Jul 17 04:48:01.384160 ip-10-0-132-19 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 17 04:48:01.384172 ip-10-0-132-19 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 17 04:48:01.384179 ip-10-0-132-19 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 17 04:48:01.384518 ip-10-0-132-19 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 17 04:48:11.633830 ip-10-0-132-19 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 17 04:48:11.633846 ip-10-0-132-19 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot fa101e777689427da7d00dc33efa46bc --
Jul 17 04:50:10.683417 ip-10-0-132-19 systemd[1]: Starting Kubernetes Kubelet...
Jul 17 04:50:11.103664 ip-10-0-132-19 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 17 04:50:11.103664 ip-10-0-132-19 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 17 04:50:11.103664 ip-10-0-132-19 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 17 04:50:11.103664 ip-10-0-132-19 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 17 04:50:11.103664 ip-10-0-132-19 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 17 04:50:11.104735 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.104649    2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 17 04:50:11.110046 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110021    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 04:50:11.110046 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110040    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 04:50:11.110046 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110044    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 04:50:11.110046 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110047    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 04:50:11.110046 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110050    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 04:50:11.110046 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110053    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 04:50:11.110283 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110057    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 04:50:11.110283 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110060    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 04:50:11.110283 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110063    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 04:50:11.110283 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110067    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 04:50:11.110283 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110071    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 04:50:11.110283 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110075    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 04:50:11.110283 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110077    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 04:50:11.110283 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110080    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 04:50:11.110283 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110083    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 04:50:11.110283 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110086    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 04:50:11.110283 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110088    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 04:50:11.110283 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110091    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 04:50:11.110283 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110093    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 04:50:11.110283 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110096    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 04:50:11.110283 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110099    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 04:50:11.110283 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110101    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 04:50:11.110283 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110104    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 04:50:11.110283 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110106    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 04:50:11.110283 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110108    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 04:50:11.110777 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110111    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 04:50:11.110777 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110114    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 04:50:11.110777 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110116    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 04:50:11.110777 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110119    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 04:50:11.110777 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110121    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 04:50:11.110777 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110125    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 04:50:11.110777 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110128    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 04:50:11.110777 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110130    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 04:50:11.110777 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110133    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 04:50:11.110777 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110152    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 04:50:11.110777 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110156    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 04:50:11.110777 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110158    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 04:50:11.110777 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110161    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 04:50:11.110777 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110164    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 04:50:11.110777 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110166    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 04:50:11.110777 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110169    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 04:50:11.110777 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110172    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 04:50:11.110777 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110174    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 04:50:11.110777 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110177    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 04:50:11.111252 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110179    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 04:50:11.111252 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110182    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 04:50:11.111252 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110186    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 04:50:11.111252 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110190    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 04:50:11.111252 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110193    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 04:50:11.111252 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110196    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 04:50:11.111252 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110198    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 04:50:11.111252 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110201    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 04:50:11.111252 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110204    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 04:50:11.111252 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110207    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 04:50:11.111252 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110209    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 04:50:11.111252 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110212    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 04:50:11.111252 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110215    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 04:50:11.111252 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110217    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 04:50:11.111252 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110220    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 04:50:11.111252 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110222    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 04:50:11.111252 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110225    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 04:50:11.111252 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110227    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 04:50:11.111252 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110230    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 04:50:11.111710 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110233    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 04:50:11.111710 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110235    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 04:50:11.111710 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110237    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 04:50:11.111710 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110240    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 04:50:11.111710 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110242    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 04:50:11.111710 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110246    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 04:50:11.111710 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110248    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 04:50:11.111710 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110252    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 04:50:11.111710 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110255    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 04:50:11.111710 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110258    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 04:50:11.111710 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110261    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 04:50:11.111710 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110263    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 04:50:11.111710 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110265    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 04:50:11.111710 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110268    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 04:50:11.111710 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110271    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 04:50:11.111710 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110274    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 04:50:11.111710 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110276    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 04:50:11.111710 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110278    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 04:50:11.111710 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110281    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 04:50:11.111710 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110283    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 04:50:11.112312 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110287    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 04:50:11.112312 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110289    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 04:50:11.112312 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110292    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 04:50:11.112312 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110295    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 04:50:11.112312 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110298    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 04:50:11.112312 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110300    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 04:50:11.112312 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110303    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 04:50:11.112312 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110305    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 04:50:11.112312 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110308    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 04:50:11.112312 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110310    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 04:50:11.112312 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110314    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 04:50:11.112312 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110317    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 04:50:11.112312 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110320    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 04:50:11.112312 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110323    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 04:50:11.112312 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110326    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 04:50:11.112312 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.110328    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 04:50:11.112312 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111041    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 04:50:11.112312 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111048    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 04:50:11.112312 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111052    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 04:50:11.112312 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111055    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 04:50:11.112788 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111058    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 04:50:11.112788 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111061    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 04:50:11.112788 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111064    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 04:50:11.112788 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111067    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 04:50:11.112788 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111071    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 04:50:11.112788 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111075    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 04:50:11.112788 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111078    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 04:50:11.112788 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111081    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 04:50:11.112788 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111084    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 04:50:11.112788 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111087    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 04:50:11.112788 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111090    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 04:50:11.112788 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111093    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 04:50:11.112788 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111095    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 04:50:11.112788 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111099    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 04:50:11.112788 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111101    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 04:50:11.112788 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111104    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 04:50:11.112788 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111106    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 04:50:11.112788 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111109    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 04:50:11.112788 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111112    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 04:50:11.112788 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111114    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 04:50:11.113329 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111117    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 04:50:11.113329 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111119    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 04:50:11.113329 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111122    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 04:50:11.113329 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111124    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 04:50:11.113329 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111127    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 04:50:11.113329 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111130    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 04:50:11.113329 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111132    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 04:50:11.113329 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111159    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 04:50:11.113329 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111163    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 04:50:11.113329 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111166    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 04:50:11.113329 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111169    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 04:50:11.113329 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111171    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 04:50:11.113329 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111174    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 04:50:11.113329 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111178    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 04:50:11.113329 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111180    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 04:50:11.113329 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111183    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 04:50:11.113329 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111185    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 04:50:11.113329 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111188    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 04:50:11.113329 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111190    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 04:50:11.113329 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111193    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 04:50:11.113822 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111195    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 04:50:11.113822 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111198    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 04:50:11.113822 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111200    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 04:50:11.113822 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111203    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 04:50:11.113822 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111205    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 04:50:11.113822 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111208    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 04:50:11.113822 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111214    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 04:50:11.113822 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111218    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 04:50:11.113822 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111220    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 04:50:11.113822 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111223    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 04:50:11.113822 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111227    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 04:50:11.113822 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111229    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 04:50:11.113822 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111232    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 04:50:11.113822 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111235    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 04:50:11.113822 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111237    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 04:50:11.113822 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111240    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 04:50:11.113822 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111243    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 04:50:11.113822 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111246    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 04:50:11.113822 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111249    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 04:50:11.114346 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111251    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 04:50:11.114346 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111254    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 04:50:11.114346 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111256    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 04:50:11.114346 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111259    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 04:50:11.114346 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111261    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 04:50:11.114346 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111264    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 04:50:11.114346 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111267    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 04:50:11.114346 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111269    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 04:50:11.114346 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111271    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 04:50:11.114346 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111274    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 04:50:11.114346 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111276    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 04:50:11.114346 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111279    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 04:50:11.114346 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111281    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 04:50:11.114346 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111283    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 04:50:11.114346 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111286    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 04:50:11.114346 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111288    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 04:50:11.114346 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111290    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 04:50:11.114346 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111293    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 04:50:11.114346 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111295    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 04:50:11.114346 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111298    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 04:50:11.114841 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111302    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 04:50:11.114841 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111304    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 04:50:11.114841 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111308    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 04:50:11.114841 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111310    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 04:50:11.114841 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111314    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 04:50:11.114841 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111317    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 04:50:11.114841 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111321    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 04:50:11.114841 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111324    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 04:50:11.114841 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111327    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 04:50:11.114841 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111330    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 04:50:11.114841 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111333    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 04:50:11.114841 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111336    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 04:50:11.114841 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111339    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 04:50:11.114841 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111341    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 04:50:11.114841 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111344    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 04:50:11.114841 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111347    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 04:50:11.114841 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111729    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 04:50:11.114841 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111734    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 04:50:11.114841 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111738    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 04:50:11.114841 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111740    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 04:50:11.115354 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111743    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 04:50:11.115354 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111746    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 04:50:11.115354 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111748    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 04:50:11.115354 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111751    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 04:50:11.115354 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111753    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 04:50:11.115354 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111756    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 04:50:11.115354 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111758    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 04:50:11.115354 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111760    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 04:50:11.115354 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111763    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 04:50:11.115354 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111765    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 04:50:11.115354 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111768    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 04:50:11.115354 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111771    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 04:50:11.115354 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111775    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 04:50:11.115354 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111779    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 04:50:11.115354 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111782    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 04:50:11.115354 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111785    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 04:50:11.115354 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111788    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 04:50:11.115354 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111792    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 04:50:11.115354 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111795    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 04:50:11.115812 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111798    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 04:50:11.115812 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111801    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 04:50:11.115812 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111803    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 04:50:11.115812 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111806    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 04:50:11.115812 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111808    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 04:50:11.115812 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111811    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 04:50:11.115812 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111813    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 04:50:11.115812 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111816    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 04:50:11.115812 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111819    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 04:50:11.115812 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111821    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 04:50:11.115812 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111824    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 04:50:11.115812 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111828    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 04:50:11.115812 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111831    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 04:50:11.115812 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111834    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 04:50:11.115812 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111836    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 04:50:11.115812 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111839    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 04:50:11.115812 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111841    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 04:50:11.115812 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111844    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 04:50:11.115812 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111847    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 04:50:11.116309 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111849    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 04:50:11.116309 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111852    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 04:50:11.116309 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111854    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 04:50:11.116309 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111857    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 04:50:11.116309 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111859    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 04:50:11.116309 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111862    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 04:50:11.116309 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111864    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 04:50:11.116309 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111868    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 04:50:11.116309 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111886    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 04:50:11.116309 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111890    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 04:50:11.116309 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111893    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 04:50:11.116309 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111897    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 04:50:11.116309 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111900    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 04:50:11.116309 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111902    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 04:50:11.116309 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111905    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 04:50:11.116309 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111908    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 04:50:11.116309 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111911    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 04:50:11.116309 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111913    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 04:50:11.116309 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111916    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 04:50:11.116309 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111918    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 04:50:11.116792 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111920    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 04:50:11.116792 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111923    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 04:50:11.116792 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111926    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 04:50:11.116792 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111928    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 04:50:11.116792 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111931    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 04:50:11.116792 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111933    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 04:50:11.116792 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111936    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 04:50:11.116792 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111938    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 04:50:11.116792 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111941    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 04:50:11.116792 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111943    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 04:50:11.116792 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111945    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 04:50:11.116792 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111947    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 04:50:11.116792 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111950    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 04:50:11.116792 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111952    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 04:50:11.116792 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111955    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 04:50:11.116792 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111957    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 04:50:11.116792 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111960    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 04:50:11.116792 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111962    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 04:50:11.116792 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111965    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 04:50:11.116792 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111967    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 04:50:11.117284 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111970    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 04:50:11.117284 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111974    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 04:50:11.117284 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111977    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 04:50:11.117284 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111979    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 04:50:11.117284 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111982    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 04:50:11.117284 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111985    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 04:50:11.117284 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111987    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 04:50:11.117284 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111990    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 04:50:11.117284 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111992    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 04:50:11.117284 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111995    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 04:50:11.117284 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.111997    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 04:50:11.117284 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112000    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 04:50:11.117284 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112002    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 04:50:11.117284 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112004    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 04:50:11.117284 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112007    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 04:50:11.117284 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112009    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 04:50:11.117284 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112012    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 04:50:11.117284 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112077    2562 flags.go:64] FLAG: --address="0.0.0.0"
Jul 17 04:50:11.117284 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112084    2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 17 04:50:11.117284 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112090    2562 flags.go:64] FLAG: --anonymous-auth="true"
Jul 17 04:50:11.117794 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112094    2562 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 17 04:50:11.117794 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112100    2562 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 17 04:50:11.117794 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112105    2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 17 04:50:11.117794 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112111    2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 17 04:50:11.117794 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112117    2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 17 04:50:11.117794 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112120    2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 17 04:50:11.117794 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112123    2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 17 04:50:11.117794 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112129    2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 17 04:50:11.117794 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112133    2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 17 04:50:11.117794 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112154    2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 17 04:50:11.117794 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112158    2562 flags.go:64] FLAG: --cgroup-root=""
Jul 17 04:50:11.117794 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112161    2562 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 17 04:50:11.117794 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112164    2562 flags.go:64] FLAG: --client-ca-file=""
Jul 17 04:50:11.117794 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112168    2562 flags.go:64] FLAG: --cloud-provider="external"
Jul 17 04:50:11.117794 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112171    2562 flags.go:64] FLAG: --cluster-dns="[]"
Jul 17 04:50:11.117794 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112176    2562 flags.go:64] FLAG: --cluster-domain=""
Jul 17 04:50:11.117794 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112179    2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 17 04:50:11.117794 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112182    2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 17 04:50:11.117794 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112185    2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 17 04:50:11.117794 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112189    2562 flags.go:64] FLAG: --container-log-max-files="5"
Jul 17 04:50:11.117794 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112193    2562 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 17 04:50:11.117794 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112196    2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 17 04:50:11.117794 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112198    2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 17 04:50:11.118387 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112212    2562 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 17 04:50:11.118387 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112215    2562 flags.go:64] FLAG: --contention-profiling="false"
Jul 17 04:50:11.118387 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112218    2562 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 17 04:50:11.118387 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112221    2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 17 04:50:11.118387 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112225    2562 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 17 04:50:11.118387 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112228    2562 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 17 04:50:11.118387 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112232    2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 17 04:50:11.118387 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112236    2562 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 17 04:50:11.118387 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112239    2562 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 17 04:50:11.118387 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112242    2562 flags.go:64] FLAG: --enable-load-reader="false"
Jul 17 04:50:11.118387 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112245    2562 flags.go:64] FLAG: --enable-server="true"
Jul 17 04:50:11.118387 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112248    2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 17 04:50:11.118387 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112253    2562 flags.go:64] FLAG: --event-burst="100"
Jul 17 04:50:11.118387 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112264    2562 flags.go:64] FLAG: --event-qps="50"
Jul 17 04:50:11.118387 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112267    2562 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 17 04:50:11.118387 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112270    2562 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 17 04:50:11.118387 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112276    2562 flags.go:64] FLAG: --eviction-hard=""
Jul 17 04:50:11.118387 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112280    2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 17 04:50:11.118387 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112283    2562 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 17 04:50:11.118387 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112286    2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 17 04:50:11.118387 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112289    2562 flags.go:64] FLAG: --eviction-soft=""
Jul 17 04:50:11.118387 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112292    2562 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 17 04:50:11.118387 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112294    2562 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 17 04:50:11.118387 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112298    2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 17 04:50:11.118387 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112302    2562 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 17 04:50:11.119061 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112305    2562 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 17 04:50:11.119061 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112308    2562 flags.go:64] FLAG: --fail-swap-on="true"
Jul 17 04:50:11.119061 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112311    2562 flags.go:64] FLAG: --feature-gates=""
Jul 17 04:50:11.119061 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112314    2562 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 17 04:50:11.119061 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112318    2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 17 04:50:11.119061 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112320    2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 17 04:50:11.119061 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112323    2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 17 04:50:11.119061 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112326    2562 flags.go:64] FLAG: --healthz-port="10248"
Jul 17 04:50:11.119061 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112330    2562 flags.go:64] FLAG: --help="false"
Jul 17 04:50:11.119061 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112332    2562 flags.go:64] FLAG: --hostname-override="ip-10-0-132-19.ec2.internal"
Jul 17 04:50:11.119061 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112335    2562 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 17 04:50:11.119061 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112338    2562 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 17 04:50:11.119061 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112341    2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 17 04:50:11.119061 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112344    2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 17 04:50:11.119061 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112348    2562 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 17 04:50:11.119061 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112350    2562 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 17 04:50:11.119061 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112353    2562 flags.go:64] FLAG: --image-service-endpoint=""
Jul 17 04:50:11.119061 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112356    2562 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 17 04:50:11.119061 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112359    2562 flags.go:64] FLAG: --kube-api-burst="100"
Jul 17 04:50:11.119061 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112362    2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 17 04:50:11.119061 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112365    2562 flags.go:64] FLAG: --kube-api-qps="50"
Jul 17 04:50:11.119061 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112367    2562 flags.go:64] FLAG: --kube-reserved=""
Jul 17 04:50:11.119061 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112370    2562 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 17 04:50:11.119061 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112374    2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 17 04:50:11.119667 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112377    2562 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 17 04:50:11.119667 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112380    2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 17 04:50:11.119667 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112383    2562 flags.go:64] FLAG: --lock-file=""
Jul 17 04:50:11.119667 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112386    2562 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 17 04:50:11.119667 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112389    2562 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 17 04:50:11.119667 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112392    2562 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 17 04:50:11.119667 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112397    2562 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 17 04:50:11.119667 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112402    2562 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 17 04:50:11.119667 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112405    2562 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 17 04:50:11.119667 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112408    2562 flags.go:64] FLAG: --logging-format="text"
Jul 17 04:50:11.119667 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112411    2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 17 04:50:11.119667 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112414    2562 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 17 04:50:11.119667 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112417    2562 flags.go:64] FLAG: --manifest-url=""
Jul 17 04:50:11.119667 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112419    2562 flags.go:64] FLAG: --manifest-url-header=""
Jul 17 04:50:11.119667 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112423    2562 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 17 04:50:11.119667 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112426    2562 flags.go:64] FLAG: --max-open-files="1000000"
Jul 17 04:50:11.119667 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112430    2562 flags.go:64] FLAG: --max-pods="110"
Jul 17 04:50:11.119667 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112433    2562 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 17 04:50:11.119667 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112435    2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 17 04:50:11.119667 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112438    2562 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 17 04:50:11.119667 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112441    2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 17 04:50:11.119667 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112444    2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 17 04:50:11.119667 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112447    2562 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 17 04:50:11.119667 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112449    2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 17 04:50:11.119667 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112457    2562 flags.go:64] FLAG: --node-status-max-images="50"
Jul 17 04:50:11.120300 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112459    2562 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 17 04:50:11.120300 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112462    2562 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 17 04:50:11.120300 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112465    2562 flags.go:64] FLAG: --pod-cidr=""
Jul 17 04:50:11.120300 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112468    2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 17 04:50:11.120300 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112473    2562 flags.go:64] FLAG: --pod-manifest-path=""
Jul 17 04:50:11.120300 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112476    2562 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 17 04:50:11.120300 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112480    2562 flags.go:64] FLAG: --pods-per-core="0"
Jul 17 04:50:11.120300 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112483    2562 flags.go:64] FLAG: --port="10250"
Jul 17 04:50:11.120300 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112485    2562 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 17 04:50:11.120300 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112488    2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0147e044747108410"
Jul 17 04:50:11.120300 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112491    2562 flags.go:64] FLAG: --qos-reserved=""
Jul 17 04:50:11.120300 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112494    2562 flags.go:64] FLAG: --read-only-port="10255"
Jul 17 04:50:11.120300 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112497    2562 flags.go:64] FLAG: --register-node="true"
Jul 17 04:50:11.120300 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112500    2562 flags.go:64] FLAG: --register-with-taints=""
Jul 17 04:50:11.120300 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112503    2562 flags.go:64] FLAG: --registry-burst="10"
Jul 17 04:50:11.120300 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112507    2562 flags.go:64] FLAG: --registry-qps="5"
Jul 17 04:50:11.120300 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112510    2562 flags.go:64] FLAG: --reserved-cpus=""
Jul 17 04:50:11.120300 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112513    2562 flags.go:64] FLAG: --reserved-memory=""
Jul 17 04:50:11.120300 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112517    2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 17 04:50:11.120300 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112520    2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 17 04:50:11.120300 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112522    2562 flags.go:64] FLAG: --rotate-certificates="false"
Jul 17 04:50:11.120300 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112525    2562 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 17 04:50:11.120300 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112528    2562 flags.go:64] FLAG: --runonce="false"
Jul 17 04:50:11.120300 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112530    2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 17 04:50:11.120300 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112533    2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 17 04:50:11.120902 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112536    2562 flags.go:64] FLAG: --seccomp-default="false"
Jul 17 04:50:11.120902 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112539    2562 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 17 04:50:11.120902 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112541    2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 17 04:50:11.120902 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112544    2562 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 17 04:50:11.120902 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112547    2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 17 04:50:11.120902 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112550    2562 flags.go:64] FLAG: --storage-driver-password="root"
Jul 17 04:50:11.120902 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112553    2562 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 17 04:50:11.120902 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112556    2562 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 17 04:50:11.120902 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112559    2562 flags.go:64] FLAG: --storage-driver-user="root"
Jul 17 04:50:11.120902 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112562    2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 17 04:50:11.120902 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112565    2562 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 17 04:50:11.120902 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112568    2562 flags.go:64] FLAG: --system-cgroups=""
Jul 17 04:50:11.120902 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112571    2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 17 04:50:11.120902 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112576    2562 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 17 04:50:11.120902 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112579    2562 flags.go:64] FLAG: --tls-cert-file=""
Jul 17 04:50:11.120902 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112582    2562 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 17 04:50:11.120902 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112585    2562 flags.go:64] FLAG: --tls-min-version=""
Jul 17 04:50:11.120902 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112588    2562 flags.go:64] FLAG: --tls-private-key-file=""
Jul 17 04:50:11.120902 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112591    2562 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 17 04:50:11.120902 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112593    2562 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 17 04:50:11.120902 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112596    2562 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 17 04:50:11.120902 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112600    2562 flags.go:64] FLAG: --v="2"
Jul 17 04:50:11.120902 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112605    2562 flags.go:64] FLAG: --version="false"
Jul 17 04:50:11.120902 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112609    2562 flags.go:64] FLAG: --vmodule=""
Jul 17 04:50:11.120902 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112613    2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 17 04:50:11.121519 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.112616    2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 17 04:50:11.121519 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112728    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 04:50:11.121519 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112731    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 04:50:11.121519 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112734    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 04:50:11.121519 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112737    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 04:50:11.121519 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112739    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 04:50:11.121519 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112742    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 04:50:11.121519 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112745    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 04:50:11.121519 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112747    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 04:50:11.121519 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112750    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 04:50:11.121519 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112752    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 04:50:11.121519 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112755    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 04:50:11.121519 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112757    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 04:50:11.121519 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112760    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 04:50:11.121519 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112762    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 04:50:11.121519 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112765    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 04:50:11.121519 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112767    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 04:50:11.121519 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112770    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 04:50:11.121519 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112772    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 04:50:11.121519 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112775    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 04:50:11.122034 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112779    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 04:50:11.122034 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112781    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 04:50:11.122034 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112783    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 04:50:11.122034 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112786    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 04:50:11.122034 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112788    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 04:50:11.122034 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112791    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 04:50:11.122034 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112794    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 04:50:11.122034 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112797    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 04:50:11.122034 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112799    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 04:50:11.122034 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112803    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 04:50:11.122034 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112806    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 04:50:11.122034 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112809    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 04:50:11.122034 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112811    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 04:50:11.122034 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112814    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 04:50:11.122034 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112816    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 04:50:11.122034 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112819    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 04:50:11.122034 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112821    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 04:50:11.122034 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112824    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 04:50:11.122034 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112826    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 04:50:11.122034 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112829    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 04:50:11.122548 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112831    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 04:50:11.122548 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112834    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 04:50:11.122548 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112836    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 04:50:11.122548 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112839    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 04:50:11.122548 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112842    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 04:50:11.122548 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112844    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 04:50:11.122548 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112846    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 04:50:11.122548 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112849    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 04:50:11.122548 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112851    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 04:50:11.122548 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112854    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 04:50:11.122548 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112857    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 04:50:11.122548 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112859    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 04:50:11.122548 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112862    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 04:50:11.122548 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112865    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 04:50:11.122548 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112867    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 04:50:11.122548 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112870    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 04:50:11.122548 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112872    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 04:50:11.122548 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112874    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 04:50:11.122548 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112878    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 04:50:11.123015 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112882    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 04:50:11.123015 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112885    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 04:50:11.123015 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112889    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 04:50:11.123015 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112892    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 04:50:11.123015 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112895    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 04:50:11.123015 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112898    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 04:50:11.123015 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112901    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 04:50:11.123015 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112904    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 04:50:11.123015 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112907    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 04:50:11.123015 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112909    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 04:50:11.123015 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112912    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 04:50:11.123015 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112916    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 04:50:11.123015 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112919    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 04:50:11.123015 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112922    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 04:50:11.123015 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112925    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 04:50:11.123015 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112928    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 04:50:11.123015 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112930    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 04:50:11.123015 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112933    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 04:50:11.123015 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112936    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 04:50:11.123488 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112938    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 04:50:11.123488 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112941    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 04:50:11.123488 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112944    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 04:50:11.123488 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112946    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 04:50:11.123488 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112949    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 04:50:11.123488 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112951    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 04:50:11.123488 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112955    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 04:50:11.123488 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112957    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 04:50:11.123488 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112959    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 04:50:11.123488 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112963    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 04:50:11.123488 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112966    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 04:50:11.123488 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112969    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 04:50:11.123488 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112971    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 04:50:11.123488 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112974    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 04:50:11.123488 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112976    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 04:50:11.123488 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112980    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 04:50:11.123488 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112982    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 04:50:11.123488 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112985    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 04:50:11.123488 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112987    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 04:50:11.123960 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112991    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 04:50:11.123960 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112993    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 04:50:11.123960 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.112996    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 04:50:11.123960 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.113623    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 17 04:50:11.123960 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.120051    2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 17 04:50:11.123960 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.120067    2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 17 04:50:11.123960 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.120084    2562 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 17 04:50:11.123960 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.120091    2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 17 04:50:11.123960 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120160    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 04:50:11.123960 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120165    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 04:50:11.123960 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120168    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 04:50:11.123960 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120171    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 04:50:11.123960 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120174    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 04:50:11.123960 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120176    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 04:50:11.123960 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120179    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 04:50:11.124352 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120182    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 04:50:11.124352 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120185    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 04:50:11.124352 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120188    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 04:50:11.124352 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120190    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 04:50:11.124352 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120194    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 04:50:11.124352 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120196    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 04:50:11.124352 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120199    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 04:50:11.124352 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120202    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 04:50:11.124352 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120205    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 04:50:11.124352 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120208    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 04:50:11.124352 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120210    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 04:50:11.124352 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120213    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 04:50:11.124352 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120216    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 04:50:11.124352 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120218    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 04:50:11.124352 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120221    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 04:50:11.124352 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120223    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 04:50:11.124352 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120226    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 04:50:11.124352 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120229    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 04:50:11.124352 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120231    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 04:50:11.124352 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120234    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 04:50:11.124825 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120236    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 04:50:11.124825 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120240    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 04:50:11.124825 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120245    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 04:50:11.124825 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120248    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 04:50:11.124825 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120250    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 04:50:11.124825 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120253    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 04:50:11.124825 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120265    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 04:50:11.124825 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120268    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 04:50:11.124825 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120271    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 04:50:11.124825 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120274    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 04:50:11.124825 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120276    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 04:50:11.124825 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120279    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 04:50:11.124825 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120282    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 04:50:11.124825 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120286    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 04:50:11.124825 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120290    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 04:50:11.124825 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120292    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 04:50:11.124825 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120295    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 04:50:11.124825 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120297    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 04:50:11.124825 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120300    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 04:50:11.125299 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120303    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 04:50:11.125299 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120305    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 04:50:11.125299 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120308    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 04:50:11.125299 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120310    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 04:50:11.125299 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120313    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 04:50:11.125299 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120316    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 04:50:11.125299 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120318    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 04:50:11.125299 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120321    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 04:50:11.125299 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120323    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 04:50:11.125299 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120326    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 04:50:11.125299 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120328    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 04:50:11.125299 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120331    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 04:50:11.125299 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120333    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 04:50:11.125299 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120336    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 04:50:11.125299 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120338    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 04:50:11.125299 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120341    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 04:50:11.125299 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120344    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 04:50:11.125299 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120347    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 04:50:11.125299 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120349    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 04:50:11.125299 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120352    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 04:50:11.125816 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120354    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 04:50:11.125816 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120356    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 04:50:11.125816 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120359    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 04:50:11.125816 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120361    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 04:50:11.125816 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120363    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 04:50:11.125816 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120366    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 04:50:11.125816 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120369    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 04:50:11.125816 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120371    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 04:50:11.125816 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120373    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 04:50:11.125816 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120376    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 04:50:11.125816 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120378    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 04:50:11.125816 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120381    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 04:50:11.125816 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120383    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 04:50:11.125816 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120386    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 04:50:11.125816 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120388    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 04:50:11.125816 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120391    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 04:50:11.125816 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120393    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 04:50:11.125816 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120396    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 04:50:11.125816 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120400    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 04:50:11.126323 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120403    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 04:50:11.126323 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120406    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 04:50:11.126323 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120409    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 04:50:11.126323 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120412    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 04:50:11.126323 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120414    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 04:50:11.126323 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120417    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 04:50:11.126323 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120420    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 04:50:11.126323 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120423    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 04:50:11.126323 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120425    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 04:50:11.126323 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120427    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 04:50:11.126323 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120430    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 04:50:11.126323 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120433    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 04:50:11.126323 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120435    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 04:50:11.126323 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120438    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 04:50:11.126665 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.120442    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 17 04:50:11.126665 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120554    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 04:50:11.126665 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120558    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 04:50:11.126665 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120560    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 04:50:11.126665 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120563    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 04:50:11.126665 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120565    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 04:50:11.126665 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120568    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 04:50:11.126665 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120570    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 04:50:11.126665 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120573    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 04:50:11.126665 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120575    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 04:50:11.126665 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120577    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 04:50:11.126665 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120580    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 04:50:11.126665 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120582    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 04:50:11.126665 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120585    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 04:50:11.126665 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120587    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 04:50:11.127017 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120590    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 04:50:11.127017 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120592    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 04:50:11.127017 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120594    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 04:50:11.127017 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120597    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 04:50:11.127017 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120599    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 04:50:11.127017 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120601    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 04:50:11.127017 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120603    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 04:50:11.127017 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120606    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 04:50:11.127017 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120609    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 04:50:11.127017 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120611    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 04:50:11.127017 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120613    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 04:50:11.127017 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120616    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 04:50:11.127017 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120618    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 04:50:11.127017 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120620    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 04:50:11.127017 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120625    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 04:50:11.127017 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120628    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 04:50:11.127017 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120630    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 04:50:11.127017 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120633    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 04:50:11.127017 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120635    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 04:50:11.127558 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120637    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 04:50:11.127558 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120641    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 04:50:11.127558 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120644    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 04:50:11.127558 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120647    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 04:50:11.127558 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120649    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 04:50:11.127558 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120652    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 04:50:11.127558 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120654    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 04:50:11.127558 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120656    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 04:50:11.127558 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120659    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 04:50:11.127558 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120662    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 04:50:11.127558 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120664    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 04:50:11.127558 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120666    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 04:50:11.127558 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120669    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 04:50:11.127558 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120672    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 04:50:11.127558 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120674    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 04:50:11.127558 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120677    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 04:50:11.127558 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120679    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 04:50:11.127558 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120681    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 04:50:11.127558 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120684    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 04:50:11.128384 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120686    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 04:50:11.128384 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120689    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 04:50:11.128384 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120691    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 04:50:11.128384 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120694    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 04:50:11.128384 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120696    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 04:50:11.128384 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120699    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 04:50:11.128384 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120701    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 04:50:11.128384 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120703    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 04:50:11.128384 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120706    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 04:50:11.128384 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120709    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 04:50:11.128384 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120711    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 04:50:11.128384 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120713    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 04:50:11.128384 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120716    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 04:50:11.128384 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120719    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 04:50:11.128384 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120721    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 04:50:11.128384 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120723    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 04:50:11.128384 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120726    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 04:50:11.128384 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120728    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 04:50:11.128384 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120730    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 04:50:11.128384 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120733    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 04:50:11.128960 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120735    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 04:50:11.128960 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120738    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 04:50:11.128960 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120741    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 04:50:11.128960 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120743    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 04:50:11.128960 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120746    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 04:50:11.128960 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120750    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 04:50:11.128960 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120753    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 04:50:11.128960 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120755    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 04:50:11.128960 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120758    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 04:50:11.128960 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120761    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 04:50:11.128960 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120763    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 04:50:11.128960 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120766    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 04:50:11.128960 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120769    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 04:50:11.128960 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120772    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 04:50:11.128960 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120775    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 04:50:11.128960 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120777    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 04:50:11.128960 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120779    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 04:50:11.128960 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120782    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 04:50:11.128960 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120784    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 04:50:11.129445 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120786    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 04:50:11.129445 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120789    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 04:50:11.129445 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120791    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 04:50:11.129445 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120794    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 04:50:11.129445 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120797    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 04:50:11.129445 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120799    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 04:50:11.129445 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120802    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 04:50:11.129445 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:11.120804    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 04:50:11.129445 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.120809    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 17 04:50:11.129445 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.120909    2562 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 17 04:50:11.129445 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.122727    2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 17 04:50:11.129445 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.124027    2562 server.go:1019] "Starting client certificate rotation"
Jul 17 04:50:11.129445 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.124128    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 17 04:50:11.129445 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.124174    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 17 04:50:11.129866 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.126341    2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 17 04:50:11.129866 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.127565    2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 17 04:50:11.137501 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.137485    2562 log.go:25] "Validated CRI v1 runtime API"
Jul 17 04:50:11.139426 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.139411    2562 log.go:25] "Validated CRI v1 image API"
Jul 17 04:50:11.141007 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.140988    2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 17 04:50:11.146088 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.146062    2562 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 995bb482-b148-4b83-9b77-16aaceaddd6e:/dev/nvme0n1p3 9bcd0e53-0462-4d2c-a6fd-d88baa7be607:/dev/nvme0n1p4]
Jul 17 04:50:11.146186 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.146088    2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 17 04:50:11.151986 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.151961    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 17 04:50:11.152229 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.152096    2562 manager.go:216] Machine: {Timestamp:2026-07-17 04:50:11.150743637 +0000 UTC m=+0.348160428 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3109100 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec227dc5a7469e3c6d7102ef7dca3c42 SystemUUID:ec227dc5-a746-9e3c-6d71-02ef7dca3c42 BootID:fa101e77-7689-427d-a7d0-0dc33efa46bc Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:da:30:69:c9:e3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:da:30:69:c9:e3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:de:ac:37:73:97:a8 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 17 04:50:11.152229 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.152226    2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 17 04:50:11.152340 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.152318    2562 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 17 04:50:11.155441 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.155415    2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 17 04:50:11.155583 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.155443    2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-19.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 17 04:50:11.156179 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.156169    2562 topology_manager.go:138] "Creating topology manager with none policy"
Jul 17 04:50:11.156216 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.156182    2562 container_manager_linux.go:309] "Creating device plugin manager"
Jul 17 04:50:11.156216 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.156190    2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 17 04:50:11.156789 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.156779    2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 17 04:50:11.158612 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.158602    2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 17 04:50:11.161223 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.161212    2562 state_mem.go:36] "Initialized new in-memory state store"
Jul 17 04:50:11.161331 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.161321    2562 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 17 04:50:11.161371 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.161350    2562 kubelet.go:486] "Attempting to sync node with API server"
Jul 17 04:50:11.161371 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.161360    2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 17 04:50:11.161442 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.161373    2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 17 04:50:11.161442 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.161382    2562 kubelet.go:398] "Adding apiserver pod source"
Jul 17 04:50:11.161442 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.161391    2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 17 04:50:11.161785 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.161771    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 17 04:50:11.161818 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.161795    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 17 04:50:11.164544 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.164529    2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 17 04:50:11.165807 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.165793    2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 17 04:50:11.165869 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.165825    2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 17 04:50:11.166450 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.166439    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 17 04:50:11.166493 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.166455    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 17 04:50:11.166493 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.166462    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 17 04:50:11.166493 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.166467    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 17 04:50:11.166573 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.166495    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 17 04:50:11.166573 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.166502    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 17 04:50:11.166573 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.166510    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 17 04:50:11.166573 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.166515    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 17 04:50:11.166573 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.166521    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 17 04:50:11.166573 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.166527    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 17 04:50:11.166573 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.166540    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 17 04:50:11.166573 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.166546    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 17 04:50:11.167242 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.167230    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 17 04:50:11.167298 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.167244    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 17 04:50:11.170829 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.170814    2562 server.go:1268] "Started kubelet"
Jul 17 04:50:11.170920 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.170892    2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 17 04:50:11.170975 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.170902    2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 17 04:50:11.171023 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.170974    2562 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 17 04:50:11.171231 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:11.171212    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 17 04:50:11.171310 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.171272    2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-19.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 17 04:50:11.171386 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:11.171370    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-19.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 17 04:50:11.171828 ip-10-0-132-19 systemd[1]: Started Kubernetes Kubelet.
Jul 17 04:50:11.175357 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.175181    2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 17 04:50:11.176487 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.176466    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cspxf"
Jul 17 04:50:11.177457 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.177441    2562 server.go:310] "Adding debug handlers to kubelet server"
Jul 17 04:50:11.177547 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:11.177534    2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 17 04:50:11.181432 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:11.180511    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-19.ec2.internal.18c2fa7b9a6b4fb9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-19.ec2.internal,UID:ip-10-0-132-19.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-19.ec2.internal,},FirstTimestamp:2026-07-17 04:50:11.170783161 +0000 UTC m=+0.368199936,LastTimestamp:2026-07-17 04:50:11.170783161 +0000 UTC m=+0.368199936,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-19.ec2.internal,}"
Jul 17 04:50:11.183402 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.183380    2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 17 04:50:11.183402 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.183396    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 17 04:50:11.183553 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:11.183503    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-19.ec2.internal\" not found"
Jul 17 04:50:11.184090 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.184071    2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 17 04:50:11.184201 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.184073    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cspxf"
Jul 17 04:50:11.184201 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.184184    2562 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 17 04:50:11.184201 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.184196    2562 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 17 04:50:11.184355 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.184325    2562 reconstruct.go:97] "Volume reconstruction finished"
Jul 17 04:50:11.184355 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.184333    2562 reconciler.go:29] "Reconciler: start to sync state"
Jul 17 04:50:11.184465 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.184447    2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 17 04:50:11.184523 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.184467    2562 factory.go:55] Registering systemd factory
Jul 17 04:50:11.184523 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.184476    2562 factory.go:223] Registration of the systemd container factory successfully
Jul 17 04:50:11.184746 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.184733    2562 factory.go:153] Registering CRI-O factory
Jul 17 04:50:11.184805 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.184751    2562 factory.go:223] Registration of the crio container factory successfully
Jul 17 04:50:11.184805 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.184778    2562 factory.go:103] Registering Raw factory
Jul 17 04:50:11.184805 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.184792    2562 manager.go:1196] Started watching for new ooms in manager
Jul 17 04:50:11.185262 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.185248    2562 manager.go:349] Starting recovery of all containers
Jul 17 04:50:11.191728 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.191552    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 17 04:50:11.195918 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.195897    2562 manager.go:354] Recovery completed
Jul 17 04:50:11.196629 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.196610    2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 04:50:11.199087 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:11.199069    2562 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-132-19.ec2.internal\" not found" node="ip-10-0-132-19.ec2.internal"
Jul 17 04:50:11.199758 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.199747    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 04:50:11.201669 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.201650    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-19.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 04:50:11.201724 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.201693    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-19.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 04:50:11.201724 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.201707    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-19.ec2.internal" event="NodeHasSufficientPID"
Jul 17 04:50:11.202317 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.202301    2562 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 17 04:50:11.202317 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.202315    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 04:50:11.202429 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.202316    2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 17 04:50:11.202429 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.202390    2562 state_mem.go:36] "Initialized new in-memory state store"
Jul 17 04:50:11.203028 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.203013    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-19.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 04:50:11.203128 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.203043    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-19.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 04:50:11.203128 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.203057    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-19.ec2.internal" event="NodeHasSufficientPID"
Jul 17 04:50:11.203438 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.203427    2562 policy_none.go:49] "None policy: Start"
Jul 17 04:50:11.203474 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.203443    2562 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 17 04:50:11.203474 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.203456    2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 17 04:50:11.204348 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.204337    2562 policy_none.go:47] "Start"
Jul 17 04:50:11.240682 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.240668    2562 manager.go:337] "Starting Device Plugin manager"
Jul 17 04:50:11.240763 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:11.240699    2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 17 04:50:11.240763 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.240712    2562 server.go:87] "Starting device plugin registration server"
Jul 17 04:50:11.240968 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.240955    2562 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 17 04:50:11.241011 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.240970    2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 17 04:50:11.241088 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.241070    2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 17 04:50:11.241236 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.241222    2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 17 04:50:11.241236 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.241233    2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 17 04:50:11.241721 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:11.241699    2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 17 04:50:11.241814 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:11.241757    2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-19.ec2.internal\" not found"
Jul 17 04:50:11.341618 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.341586    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 04:50:11.342677 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.342658    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-19.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 04:50:11.342736 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.342700    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-19.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 04:50:11.342736 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.342712    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-19.ec2.internal" event="NodeHasSufficientPID"
Jul 17 04:50:11.342811 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.342737    2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-19.ec2.internal"
Jul 17 04:50:11.350656 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.350642    2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-19.ec2.internal"
Jul 17 04:50:11.350722 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:11.350662    2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-19.ec2.internal\": node \"ip-10-0-132-19.ec2.internal\" not found"
Jul 17 04:50:11.363608 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.363557    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 17 04:50:11.363608 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.363597    2562 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 17 04:50:11.363727 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.363618    2562 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 17 04:50:11.363727 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:11.363646    2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 17 04:50:11.364725 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:11.364709    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-19.ec2.internal\" not found"
Jul 17 04:50:11.365743 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.365728    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 04:50:11.464152 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.464097    2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-19.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-19.ec2.internal"]
Jul 17 04:50:11.464286 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.464191    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 04:50:11.465599 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:11.465585    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-19.ec2.internal\" not found"
Jul 17 04:50:11.465926 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.465903    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-19.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 04:50:11.466014 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.465934    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-19.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 04:50:11.466014 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.465946    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-19.ec2.internal" event="NodeHasSufficientPID"
Jul 17 04:50:11.467267 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.467253    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 04:50:11.467398 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.467385    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-19.ec2.internal"
Jul 17 04:50:11.467448 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.467414    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 04:50:11.468029 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.468004    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-19.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 04:50:11.468118 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.468043    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-19.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 04:50:11.468118 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.468058    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-19.ec2.internal" event="NodeHasSufficientPID"
Jul 17 04:50:11.468118 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.468070    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-19.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 04:50:11.468118 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.468092    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-19.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 04:50:11.468118 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.468109    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-19.ec2.internal" event="NodeHasSufficientPID"
Jul 17 04:50:11.469868 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.469851    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-19.ec2.internal"
Jul 17 04:50:11.469941 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.469877    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 04:50:11.470575 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.470559    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-19.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 04:50:11.470657 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.470585    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-19.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 04:50:11.470657 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.470607    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-19.ec2.internal" event="NodeHasSufficientPID"
Jul 17 04:50:11.497699 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:11.497682    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-19.ec2.internal\" not found" node="ip-10-0-132-19.ec2.internal"
Jul 17 04:50:11.501999 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:11.501983    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-19.ec2.internal\" not found" node="ip-10-0-132-19.ec2.internal"
Jul 17 04:50:11.565804 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:11.565784    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-19.ec2.internal\" not found"
Jul 17 04:50:11.586151 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.586118    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/02dd81e7d773510fb8d800d8518d7fdd-config\") pod \"kube-apiserver-proxy-ip-10-0-132-19.ec2.internal\" (UID: \"02dd81e7d773510fb8d800d8518d7fdd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-19.ec2.internal"
Jul 17 04:50:11.586231 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.586156    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/aea730e39f6c59e91b6798ae3c762419-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-19.ec2.internal\" (UID: \"aea730e39f6c59e91b6798ae3c762419\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-19.ec2.internal"
Jul 17 04:50:11.586231 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.586173    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aea730e39f6c59e91b6798ae3c762419-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-19.ec2.internal\" (UID: \"aea730e39f6c59e91b6798ae3c762419\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-19.ec2.internal"
Jul 17 04:50:11.666596 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:11.666536    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-19.ec2.internal\" not found"
Jul 17 04:50:11.686866 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.686842    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/aea730e39f6c59e91b6798ae3c762419-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-19.ec2.internal\" (UID: \"aea730e39f6c59e91b6798ae3c762419\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-19.ec2.internal"
Jul 17 04:50:11.686923 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.686871    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aea730e39f6c59e91b6798ae3c762419-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-19.ec2.internal\" (UID: \"aea730e39f6c59e91b6798ae3c762419\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-19.ec2.internal"
Jul 17 04:50:11.686923 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.686907    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/02dd81e7d773510fb8d800d8518d7fdd-config\") pod \"kube-apiserver-proxy-ip-10-0-132-19.ec2.internal\" (UID: \"02dd81e7d773510fb8d800d8518d7fdd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-19.ec2.internal"
Jul 17 04:50:11.687021 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.686943    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/aea730e39f6c59e91b6798ae3c762419-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-19.ec2.internal\" (UID: \"aea730e39f6c59e91b6798ae3c762419\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-19.ec2.internal"
Jul 17 04:50:11.687021 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.686954    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/02dd81e7d773510fb8d800d8518d7fdd-config\") pod \"kube-apiserver-proxy-ip-10-0-132-19.ec2.internal\" (UID: \"02dd81e7d773510fb8d800d8518d7fdd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-19.ec2.internal"
Jul 17 04:50:11.687021 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.686974    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aea730e39f6c59e91b6798ae3c762419-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-19.ec2.internal\" (UID: \"aea730e39f6c59e91b6798ae3c762419\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-19.ec2.internal"
Jul 17 04:50:11.767026 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:11.766994    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-19.ec2.internal\" not found"
Jul 17 04:50:11.799443 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.799425    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-19.ec2.internal"
Jul 17 04:50:11.806171 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:11.806153    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-19.ec2.internal"
Jul 17 04:50:11.868030 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:11.867996    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-19.ec2.internal\" not found"
Jul 17 04:50:11.968546 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:11.968468    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-19.ec2.internal\" not found"
Jul 17 04:50:13.582782 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:13.582759    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-19.ec2.internal\" not found"
Jul 17 04:50:13.638319 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:13.638293    2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 17 04:50:13.638737 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:13.638454    2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 17 04:50:13.638737 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:13.638477    2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 17 04:50:13.683855 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:13.683819    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-19.ec2.internal\" not found"
Jul 17 04:50:13.704555 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:13.704519    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-16 04:45:12 +0000 UTC" deadline="2028-04-09 01:02:22.795713236 +0000 UTC"
Jul 17 04:50:13.704555 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:13.704544    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15164h12m9.091171665s"
Jul 17 04:50:13.784602 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:13.784518    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-19.ec2.internal\" not found"
Jul 17 04:50:13.885282 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:13.885254    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-19.ec2.internal\" not found"
Jul 17 04:50:13.932823 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:13.932793    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02dd81e7d773510fb8d800d8518d7fdd.slice/crio-e85c77ca4c43e27bad4ea7809e46f4486005bb7628bac64b1a501b0526246ddc WatchSource:0}: Error finding container e85c77ca4c43e27bad4ea7809e46f4486005bb7628bac64b1a501b0526246ddc: Status 404 returned error can't find the container with id e85c77ca4c43e27bad4ea7809e46f4486005bb7628bac64b1a501b0526246ddc
Jul 17 04:50:13.933136 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:13.933120    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaea730e39f6c59e91b6798ae3c762419.slice/crio-f068137c6abfd736c2da150a771a91bf27243f8c01d0dc94e2b50e76b826c26e WatchSource:0}: Error finding container f068137c6abfd736c2da150a771a91bf27243f8c01d0dc94e2b50e76b826c26e: Status 404 returned error can't find the container with id f068137c6abfd736c2da150a771a91bf27243f8c01d0dc94e2b50e76b826c26e
Jul 17 04:50:13.936767 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:13.936754    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 04:50:13.959313 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:13.959292    2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 04:50:13.985487 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:13.985446    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-19.ec2.internal\" not found"
Jul 17 04:50:14.059616 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.059557    2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 04:50:14.097510 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.097482    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-19.ec2.internal"
Jul 17 04:50:14.108206 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.108189    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 17 04:50:14.109126 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.109113    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-19.ec2.internal"
Jul 17 04:50:14.115022 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.115010    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 17 04:50:14.676697 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.676664    2562 apiserver.go:52] "Watching apiserver"
Jul 17 04:50:14.685246 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.685221    2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 17 04:50:14.685747 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.685717    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-bwdpn","openshift-multus/multus-lbjqq","openshift-network-operator/iptables-alerter-kfkjz","openshift-ovn-kubernetes/ovnkube-node-tx9gj","kube-system/global-pull-secret-syncer-6bkt5","kube-system/kube-apiserver-proxy-ip-10-0-132-19.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7tkkm","openshift-cluster-node-tuning-operator/tuned-chbnw","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-19.ec2.internal","openshift-multus/multus-additional-cni-plugins-644hn","openshift-multus/network-metrics-daemon-rzdct","openshift-network-diagnostics/network-check-target-9vmqd","kube-system/konnectivity-agent-nbdgc","openshift-dns/node-resolver-4l4dn"]
Jul 17 04:50:14.687365 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.687336    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-644hn"
Jul 17 04:50:14.689569 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.689539    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.691300 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.691280    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kfkjz"
Jul 17 04:50:14.692854 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.692839    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bwdpn"
Jul 17 04:50:14.693732 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.693413    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nbdgc"
Jul 17 04:50:14.693909 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.693877    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 17 04:50:14.693999 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.693934    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-dfx6f\""
Jul 17 04:50:14.694206 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.694183    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 17 04:50:14.694335 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.694323    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 17 04:50:14.694447 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.694436    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 17 04:50:14.694756 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.694739    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 17 04:50:14.694829 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.694789    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 17 04:50:14.695101 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.695085    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-58jsd\""
Jul 17 04:50:14.696205 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.695632    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 17 04:50:14.696205 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.695715    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vmqd"
Jul 17 04:50:14.696205 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:14.695814    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vmqd" podUID="bcee811d-1474-45f6-b0c7-afceee510ccd"
Jul 17 04:50:14.697210 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.697117    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-8kszg\""
Jul 17 04:50:14.697319 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.697300    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 17 04:50:14.697386 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.697318    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 17 04:50:14.697494 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.697474    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-znb26\""
Jul 17 04:50:14.697599 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.697504    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 17 04:50:14.697660 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.697639    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-kkhjl\""
Jul 17 04:50:14.697715 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.697652    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 17 04:50:14.697715 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.697675    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 17 04:50:14.697797 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.697767    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 17 04:50:14.697904 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.697887    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzdct"
Jul 17 04:50:14.698045 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.698026    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 17 04:50:14.698120 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:14.698101    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzdct" podUID="a54b09be-6889-45d6-a748-68e5b94b73d2"
Jul 17 04:50:14.698179 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.698154    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 17 04:50:14.698842 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.698821    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-chbnw"
Jul 17 04:50:14.700442 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.700424    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.701332 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.701314    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 17 04:50:14.701532 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.701513    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xrqkr\""
Jul 17 04:50:14.701853 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.701830    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 17 04:50:14.702174 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.702156    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7tkkm"
Jul 17 04:50:14.702747 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.702726    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 17 04:50:14.702845 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.702732    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 17 04:50:14.702920 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.702903    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 17 04:50:14.703898 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.702918    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 17 04:50:14.703898 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.703354    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 17 04:50:14.703898 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.703404    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-4hb6h\""
Jul 17 04:50:14.703898 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.703720    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6bkt5"
Jul 17 04:50:14.704166 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:14.703931    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6bkt5" podUID="237abccc-c19c-4fca-be94-d3013f3c2e86"
Jul 17 04:50:14.704723 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.704703    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4l4dn"
Jul 17 04:50:14.705274 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.705256    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 17 04:50:14.705424 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.705404    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 17 04:50:14.705555 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.705538    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 17 04:50:14.705741 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.705700    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 17 04:50:14.705741 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.705372    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-lxwc4\""
Jul 17 04:50:14.706772 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.706751    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 17 04:50:14.707687 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.707163    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wpst4\""
Jul 17 04:50:14.707687 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.707628    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 17 04:50:14.708215 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.708196    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 17 04:50:14.708467 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.708441    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/01097a42-523b-4764-b6a6-bf531915f32f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-644hn\" (UID: \"01097a42-523b-4764-b6a6-bf531915f32f\") " pod="openshift-multus/multus-additional-cni-plugins-644hn"
Jul 17 04:50:14.708536 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.708484    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gsgv\" (UniqueName: \"kubernetes.io/projected/01097a42-523b-4764-b6a6-bf531915f32f-kube-api-access-5gsgv\") pod \"multus-additional-cni-plugins-644hn\" (UID: \"01097a42-523b-4764-b6a6-bf531915f32f\") " pod="openshift-multus/multus-additional-cni-plugins-644hn"
Jul 17 04:50:14.708590 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.708535    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/da4d7ee5-31c8-465a-b5bd-5127ac635d08-konnectivity-ca\") pod \"konnectivity-agent-nbdgc\" (UID: \"da4d7ee5-31c8-465a-b5bd-5127ac635d08\") " pod="kube-system/konnectivity-agent-nbdgc"
Jul 17 04:50:14.708590 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.708563    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/36349201-affc-4d02-9527-4f43167cdb1e-sys\") pod \"tuned-chbnw\" (UID: \"36349201-affc-4d02-9527-4f43167cdb1e\") " pod="openshift-cluster-node-tuning-operator/tuned-chbnw"
Jul 17 04:50:14.708677 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.708595    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/36349201-affc-4d02-9527-4f43167cdb1e-etc-tuned\") pod \"tuned-chbnw\" (UID: \"36349201-affc-4d02-9527-4f43167cdb1e\") " pod="openshift-cluster-node-tuning-operator/tuned-chbnw"
Jul 17 04:50:14.708677 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.708630    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f4c22119-9902-4e1f-bc76-97d51253b16c-host-var-lib-cni-bin\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.708677 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.708652    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/01097a42-523b-4764-b6a6-bf531915f32f-cnibin\") pod \"multus-additional-cni-plugins-644hn\" (UID: \"01097a42-523b-4764-b6a6-bf531915f32f\") " pod="openshift-multus/multus-additional-cni-plugins-644hn"
Jul 17 04:50:14.708759 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.708696    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/01097a42-523b-4764-b6a6-bf531915f32f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-644hn\" (UID: \"01097a42-523b-4764-b6a6-bf531915f32f\") " pod="openshift-multus/multus-additional-cni-plugins-644hn"
Jul 17 04:50:14.708759 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.708730    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/da4d7ee5-31c8-465a-b5bd-5127ac635d08-agent-certs\") pod \"konnectivity-agent-nbdgc\" (UID: \"da4d7ee5-31c8-465a-b5bd-5127ac635d08\") " pod="kube-system/konnectivity-agent-nbdgc"
Jul 17 04:50:14.708759 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.708755    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/36349201-affc-4d02-9527-4f43167cdb1e-etc-sysctl-d\") pod \"tuned-chbnw\" (UID: \"36349201-affc-4d02-9527-4f43167cdb1e\") " pod="openshift-cluster-node-tuning-operator/tuned-chbnw"
Jul 17 04:50:14.708888 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.708778    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f4c22119-9902-4e1f-bc76-97d51253b16c-host-var-lib-kubelet\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.708888 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.708832    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f4c22119-9902-4e1f-bc76-97d51253b16c-host-run-multus-certs\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.708888 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.708876    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f4c22119-9902-4e1f-bc76-97d51253b16c-etc-kubernetes\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.709043 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.708906    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/01097a42-523b-4764-b6a6-bf531915f32f-cni-binary-copy\") pod \"multus-additional-cni-plugins-644hn\" (UID: \"01097a42-523b-4764-b6a6-bf531915f32f\") " pod="openshift-multus/multus-additional-cni-plugins-644hn"
Jul 17 04:50:14.709043 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.708931    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql6wg\" (UniqueName: \"kubernetes.io/projected/43b05be8-ccbf-412d-ad62-74596e1710fa-kube-api-access-ql6wg\") pod \"node-ca-bwdpn\" (UID: \"43b05be8-ccbf-412d-ad62-74596e1710fa\") " pod="openshift-image-registry/node-ca-bwdpn"
Jul 17 04:50:14.709043 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.708953    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/36349201-affc-4d02-9527-4f43167cdb1e-etc-systemd\") pod \"tuned-chbnw\" (UID: \"36349201-affc-4d02-9527-4f43167cdb1e\") " pod="openshift-cluster-node-tuning-operator/tuned-chbnw"
Jul 17 04:50:14.709043 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.709014    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbw2m\" (UniqueName: \"kubernetes.io/projected/36349201-affc-4d02-9527-4f43167cdb1e-kube-api-access-hbw2m\") pod \"tuned-chbnw\" (UID: \"36349201-affc-4d02-9527-4f43167cdb1e\") " pod="openshift-cluster-node-tuning-operator/tuned-chbnw"
Jul 17 04:50:14.709176 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.709054    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4c22119-9902-4e1f-bc76-97d51253b16c-multus-cni-dir\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.709176 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.709086    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f4c22119-9902-4e1f-bc76-97d51253b16c-os-release\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.709176 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.709110    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f4c22119-9902-4e1f-bc76-97d51253b16c-host-var-lib-cni-multus\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.709176 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.709133    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f4c22119-9902-4e1f-bc76-97d51253b16c-multus-conf-dir\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.709176 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.709156    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/01097a42-523b-4764-b6a6-bf531915f32f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-644hn\" (UID: \"01097a42-523b-4764-b6a6-bf531915f32f\") " pod="openshift-multus/multus-additional-cni-plugins-644hn"
Jul 17 04:50:14.709312 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.709179    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/36349201-affc-4d02-9527-4f43167cdb1e-etc-kubernetes\") pod \"tuned-chbnw\" (UID: \"36349201-affc-4d02-9527-4f43167cdb1e\") " pod="openshift-cluster-node-tuning-operator/tuned-chbnw"
Jul 17 04:50:14.709312 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.709202    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f4c22119-9902-4e1f-bc76-97d51253b16c-multus-daemon-config\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.709312 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.709234    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/14db56e5-42df-4de1-bb13-86d893eed9c0-iptables-alerter-script\") pod \"iptables-alerter-kfkjz\" (UID: \"14db56e5-42df-4de1-bb13-86d893eed9c0\") " pod="openshift-network-operator/iptables-alerter-kfkjz"
Jul 17 04:50:14.709312 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.709258    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bmqm\" (UniqueName: \"kubernetes.io/projected/a54b09be-6889-45d6-a748-68e5b94b73d2-kube-api-access-4bmqm\") pod \"network-metrics-daemon-rzdct\" (UID: \"a54b09be-6889-45d6-a748-68e5b94b73d2\") " pod="openshift-multus/network-metrics-daemon-rzdct"
Jul 17 04:50:14.709312 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.709282    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/36349201-affc-4d02-9527-4f43167cdb1e-var-lib-kubelet\") pod \"tuned-chbnw\" (UID: \"36349201-affc-4d02-9527-4f43167cdb1e\") " pod="openshift-cluster-node-tuning-operator/tuned-chbnw"
Jul 17 04:50:14.709477 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.709320    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4c22119-9902-4e1f-bc76-97d51253b16c-system-cni-dir\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.709477 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.709356    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f4c22119-9902-4e1f-bc76-97d51253b16c-host-run-netns\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.709477 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.709400    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a54b09be-6889-45d6-a748-68e5b94b73d2-metrics-certs\") pod \"network-metrics-daemon-rzdct\" (UID: \"a54b09be-6889-45d6-a748-68e5b94b73d2\") " pod="openshift-multus/network-metrics-daemon-rzdct"
Jul 17 04:50:14.709477 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.709422    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/43b05be8-ccbf-412d-ad62-74596e1710fa-host\") pod \"node-ca-bwdpn\" (UID: \"43b05be8-ccbf-412d-ad62-74596e1710fa\") " pod="openshift-image-registry/node-ca-bwdpn"
Jul 17 04:50:14.709477 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.709445    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/43b05be8-ccbf-412d-ad62-74596e1710fa-serviceca\") pod \"node-ca-bwdpn\" (UID: \"43b05be8-ccbf-412d-ad62-74596e1710fa\") " pod="openshift-image-registry/node-ca-bwdpn"
Jul 17 04:50:14.709477 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.709465    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/36349201-affc-4d02-9527-4f43167cdb1e-etc-modprobe-d\") pod \"tuned-chbnw\" (UID: \"36349201-affc-4d02-9527-4f43167cdb1e\") " pod="openshift-cluster-node-tuning-operator/tuned-chbnw"
Jul 17 04:50:14.709646 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.709504    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f4c22119-9902-4e1f-bc76-97d51253b16c-cnibin\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.709646 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.709532    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4xwm\" (UniqueName: \"kubernetes.io/projected/bcee811d-1474-45f6-b0c7-afceee510ccd-kube-api-access-l4xwm\") pod \"network-check-target-9vmqd\" (UID: \"bcee811d-1474-45f6-b0c7-afceee510ccd\") " pod="openshift-network-diagnostics/network-check-target-9vmqd"
Jul 17 04:50:14.709646 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.709569    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/01097a42-523b-4764-b6a6-bf531915f32f-system-cni-dir\") pod \"multus-additional-cni-plugins-644hn\" (UID: \"01097a42-523b-4764-b6a6-bf531915f32f\") " pod="openshift-multus/multus-additional-cni-plugins-644hn"
Jul 17 04:50:14.709646 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.709593    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/01097a42-523b-4764-b6a6-bf531915f32f-os-release\") pod \"multus-additional-cni-plugins-644hn\" (UID: \"01097a42-523b-4764-b6a6-bf531915f32f\") " pod="openshift-multus/multus-additional-cni-plugins-644hn"
Jul 17 04:50:14.709646 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.709615    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/36349201-affc-4d02-9527-4f43167cdb1e-etc-sysctl-conf\") pod \"tuned-chbnw\" (UID: \"36349201-affc-4d02-9527-4f43167cdb1e\") " pod="openshift-cluster-node-tuning-operator/tuned-chbnw"
Jul 17 04:50:14.709646 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.709635    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/36349201-affc-4d02-9527-4f43167cdb1e-lib-modules\") pod \"tuned-chbnw\" (UID: \"36349201-affc-4d02-9527-4f43167cdb1e\") " pod="openshift-cluster-node-tuning-operator/tuned-chbnw"
Jul 17 04:50:14.709805 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.709657    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/36349201-affc-4d02-9527-4f43167cdb1e-host\") pod \"tuned-chbnw\" (UID: \"36349201-affc-4d02-9527-4f43167cdb1e\") " pod="openshift-cluster-node-tuning-operator/tuned-chbnw"
Jul 17 04:50:14.709805 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.709678    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/36349201-affc-4d02-9527-4f43167cdb1e-tmp\") pod \"tuned-chbnw\" (UID: \"36349201-affc-4d02-9527-4f43167cdb1e\") " pod="openshift-cluster-node-tuning-operator/tuned-chbnw"
Jul 17 04:50:14.709805 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.709703    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f4c22119-9902-4e1f-bc76-97d51253b16c-multus-socket-dir-parent\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.709805 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.709742    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f4c22119-9902-4e1f-bc76-97d51253b16c-host-run-k8s-cni-cncf-io\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.709805 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.709766    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/36349201-affc-4d02-9527-4f43167cdb1e-etc-sysconfig\") pod \"tuned-chbnw\" (UID: \"36349201-affc-4d02-9527-4f43167cdb1e\") " pod="openshift-cluster-node-tuning-operator/tuned-chbnw"
Jul 17 04:50:14.709805 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.709788    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/36349201-affc-4d02-9527-4f43167cdb1e-run\") pod \"tuned-chbnw\" (UID: \"36349201-affc-4d02-9527-4f43167cdb1e\") " pod="openshift-cluster-node-tuning-operator/tuned-chbnw"
Jul 17 04:50:14.710007 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.709811    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f4c22119-9902-4e1f-bc76-97d51253b16c-cni-binary-copy\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.710007 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.709843    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f4c22119-9902-4e1f-bc76-97d51253b16c-hostroot\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.710007 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.709871    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-459x5\" (UniqueName: \"kubernetes.io/projected/f4c22119-9902-4e1f-bc76-97d51253b16c-kube-api-access-459x5\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.710007 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.709894    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/14db56e5-42df-4de1-bb13-86d893eed9c0-host-slash\") pod \"iptables-alerter-kfkjz\" (UID: \"14db56e5-42df-4de1-bb13-86d893eed9c0\") " pod="openshift-network-operator/iptables-alerter-kfkjz"
Jul 17 04:50:14.710007 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.709916    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqgfw\" (UniqueName: \"kubernetes.io/projected/14db56e5-42df-4de1-bb13-86d893eed9c0-kube-api-access-jqgfw\") pod \"iptables-alerter-kfkjz\" (UID: \"14db56e5-42df-4de1-bb13-86d893eed9c0\") " pod="openshift-network-operator/iptables-alerter-kfkjz"
Jul 17 04:50:14.728302 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.728279    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-c2jbk"
Jul 17 04:50:14.736394 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.736017    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-c2jbk"
Jul 17 04:50:14.739701 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.739674    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 04:50:14.798981 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.798878    2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 17 04:50:14.810731 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.810699    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/36349201-affc-4d02-9527-4f43167cdb1e-etc-sysctl-conf\") pod \"tuned-chbnw\" (UID: \"36349201-affc-4d02-9527-4f43167cdb1e\") " pod="openshift-cluster-node-tuning-operator/tuned-chbnw"
Jul 17 04:50:14.810868 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.810742    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/36349201-affc-4d02-9527-4f43167cdb1e-tmp\") pod \"tuned-chbnw\" (UID: \"36349201-affc-4d02-9527-4f43167cdb1e\") " pod="openshift-cluster-node-tuning-operator/tuned-chbnw"
Jul 17 04:50:14.810929 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.810891    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/36349201-affc-4d02-9527-4f43167cdb1e-etc-sysctl-conf\") pod \"tuned-chbnw\" (UID: \"36349201-affc-4d02-9527-4f43167cdb1e\") " pod="openshift-cluster-node-tuning-operator/tuned-chbnw"
Jul 17 04:50:14.810999 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.810930    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/36349201-affc-4d02-9527-4f43167cdb1e-run\") pod \"tuned-chbnw\" (UID: \"36349201-affc-4d02-9527-4f43167cdb1e\") " pod="openshift-cluster-node-tuning-operator/tuned-chbnw"
Jul 17 04:50:14.810999 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.810955    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f4c22119-9902-4e1f-bc76-97d51253b16c-hostroot\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.811102 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.811003    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-459x5\" (UniqueName: \"kubernetes.io/projected/f4c22119-9902-4e1f-bc76-97d51253b16c-kube-api-access-459x5\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.811102 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.811032    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c1baeee2-350c-43c5-9a04-fb34378edb5c-host-kubelet\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.811102 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.811075    2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 17 04:50:14.811102 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.811082    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f4c22119-9902-4e1f-bc76-97d51253b16c-hostroot\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.811102 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.811079    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c1baeee2-350c-43c5-9a04-fb34378edb5c-run-systemd\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.811433 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.811128    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c1baeee2-350c-43c5-9a04-fb34378edb5c-node-log\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.811433 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.811164    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/01097a42-523b-4764-b6a6-bf531915f32f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-644hn\" (UID: \"01097a42-523b-4764-b6a6-bf531915f32f\") " pod="openshift-multus/multus-additional-cni-plugins-644hn"
Jul 17 04:50:14.811433 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.811301    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/01097a42-523b-4764-b6a6-bf531915f32f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-644hn\" (UID: \"01097a42-523b-4764-b6a6-bf531915f32f\") " pod="openshift-multus/multus-additional-cni-plugins-644hn"
Jul 17 04:50:14.811433 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.811337    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/36349201-affc-4d02-9527-4f43167cdb1e-sys\") pod \"tuned-chbnw\" (UID: \"36349201-affc-4d02-9527-4f43167cdb1e\") " pod="openshift-cluster-node-tuning-operator/tuned-chbnw"
Jul 17 04:50:14.811433 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.811368    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f4c22119-9902-4e1f-bc76-97d51253b16c-host-var-lib-cni-bin\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.811433 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.811391    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/36349201-affc-4d02-9527-4f43167cdb1e-run\") pod \"tuned-chbnw\" (UID: \"36349201-affc-4d02-9527-4f43167cdb1e\") " pod="openshift-cluster-node-tuning-operator/tuned-chbnw"
Jul 17 04:50:14.811433 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.811411    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f4c22119-9902-4e1f-bc76-97d51253b16c-host-var-lib-cni-bin\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.811756 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.811444    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/36349201-affc-4d02-9527-4f43167cdb1e-sys\") pod \"tuned-chbnw\" (UID: \"36349201-affc-4d02-9527-4f43167cdb1e\") " pod="openshift-cluster-node-tuning-operator/tuned-chbnw"
Jul 17 04:50:14.811756 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.811489    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7960477c-6164-45a9-a338-3063feb4a4a2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7tkkm\" (UID: \"7960477c-6164-45a9-a338-3063feb4a4a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7tkkm"
Jul 17 04:50:14.811756 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.811517    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c1baeee2-350c-43c5-9a04-fb34378edb5c-ovnkube-config\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.811756 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.811541    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/237abccc-c19c-4fca-be94-d3013f3c2e86-original-pull-secret\") pod \"global-pull-secret-syncer-6bkt5\" (UID: \"237abccc-c19c-4fca-be94-d3013f3c2e86\") " pod="kube-system/global-pull-secret-syncer-6bkt5"
Jul 17 04:50:14.811756 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.811567    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7960477c-6164-45a9-a338-3063feb4a4a2-socket-dir\") pod \"aws-ebs-csi-driver-node-7tkkm\" (UID: \"7960477c-6164-45a9-a338-3063feb4a4a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7tkkm"
Jul 17 04:50:14.811756 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.811597    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/01097a42-523b-4764-b6a6-bf531915f32f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-644hn\" (UID: \"01097a42-523b-4764-b6a6-bf531915f32f\") " pod="openshift-multus/multus-additional-cni-plugins-644hn"
Jul 17 04:50:14.811756 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.811623    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/36349201-affc-4d02-9527-4f43167cdb1e-etc-sysctl-d\") pod \"tuned-chbnw\" (UID: \"36349201-affc-4d02-9527-4f43167cdb1e\") " pod="openshift-cluster-node-tuning-operator/tuned-chbnw"
Jul 17 04:50:14.811756 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.811649    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f4c22119-9902-4e1f-bc76-97d51253b16c-host-run-multus-certs\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.811756 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.811674    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c1baeee2-350c-43c5-9a04-fb34378edb5c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.811756 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.811702    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c1baeee2-350c-43c5-9a04-fb34378edb5c-ovn-node-metrics-cert\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.811756 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.811724    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jt92\" (UniqueName: \"kubernetes.io/projected/c1baeee2-350c-43c5-9a04-fb34378edb5c-kube-api-access-4jt92\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.811756 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.811750    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8ptp\" (UniqueName: \"kubernetes.io/projected/052c1550-58cb-4558-8b40-d9bd51a597b9-kube-api-access-r8ptp\") pod \"node-resolver-4l4dn\" (UID: \"052c1550-58cb-4558-8b40-d9bd51a597b9\") " pod="openshift-dns/node-resolver-4l4dn"
Jul 17 04:50:14.812322 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.811773    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f4c22119-9902-4e1f-bc76-97d51253b16c-os-release\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.812322 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.811797    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f4c22119-9902-4e1f-bc76-97d51253b16c-multus-conf-dir\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.812322 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.811820    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f4c22119-9902-4e1f-bc76-97d51253b16c-multus-daemon-config\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.812322 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.811842    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c1baeee2-350c-43c5-9a04-fb34378edb5c-etc-openvswitch\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.812322 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.811868    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c1baeee2-350c-43c5-9a04-fb34378edb5c-host-run-ovn-kubernetes\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.812322 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.811892    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c1baeee2-350c-43c5-9a04-fb34378edb5c-host-slash\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.812322 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.811917    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/36349201-affc-4d02-9527-4f43167cdb1e-var-lib-kubelet\") pod \"tuned-chbnw\" (UID: \"36349201-affc-4d02-9527-4f43167cdb1e\") " pod="openshift-cluster-node-tuning-operator/tuned-chbnw"
Jul 17 04:50:14.812322 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.811941    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a54b09be-6889-45d6-a748-68e5b94b73d2-metrics-certs\") pod \"network-metrics-daemon-rzdct\" (UID: \"a54b09be-6889-45d6-a748-68e5b94b73d2\") " pod="openshift-multus/network-metrics-daemon-rzdct"
Jul 17 04:50:14.812322 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.811988    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7960477c-6164-45a9-a338-3063feb4a4a2-device-dir\") pod \"aws-ebs-csi-driver-node-7tkkm\" (UID: \"7960477c-6164-45a9-a338-3063feb4a4a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7tkkm"
Jul 17 04:50:14.812322 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.812028    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c1baeee2-350c-43c5-9a04-fb34378edb5c-run-ovn\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.812322 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.812057    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c1baeee2-350c-43c5-9a04-fb34378edb5c-host-cni-netd\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.812322 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.812076    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/01097a42-523b-4764-b6a6-bf531915f32f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-644hn\" (UID: \"01097a42-523b-4764-b6a6-bf531915f32f\") " pod="openshift-multus/multus-additional-cni-plugins-644hn"
Jul 17 04:50:14.812322 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.812082    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/43b05be8-ccbf-412d-ad62-74596e1710fa-host\") pod \"node-ca-bwdpn\" (UID: \"43b05be8-ccbf-412d-ad62-74596e1710fa\") " pod="openshift-image-registry/node-ca-bwdpn"
Jul 17 04:50:14.812322 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.812106    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/43b05be8-ccbf-412d-ad62-74596e1710fa-serviceca\") pod \"node-ca-bwdpn\" (UID: \"43b05be8-ccbf-412d-ad62-74596e1710fa\") " pod="openshift-image-registry/node-ca-bwdpn"
Jul 17 04:50:14.812322 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.812132    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c1baeee2-350c-43c5-9a04-fb34378edb5c-run-openvswitch\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.812322 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.812183    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c1baeee2-350c-43c5-9a04-fb34378edb5c-ovnkube-script-lib\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.813111 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.812331    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/36349201-affc-4d02-9527-4f43167cdb1e-etc-sysctl-d\") pod \"tuned-chbnw\" (UID: \"36349201-affc-4d02-9527-4f43167cdb1e\") " pod="openshift-cluster-node-tuning-operator/tuned-chbnw"
Jul 17 04:50:14.813111 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.812382    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f4c22119-9902-4e1f-bc76-97d51253b16c-host-run-multus-certs\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.813111 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.812392    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/43b05be8-ccbf-412d-ad62-74596e1710fa-host\") pod \"node-ca-bwdpn\" (UID: \"43b05be8-ccbf-412d-ad62-74596e1710fa\") " pod="openshift-image-registry/node-ca-bwdpn"
Jul 17 04:50:14.813111 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.812435    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/36349201-affc-4d02-9527-4f43167cdb1e-var-lib-kubelet\") pod \"tuned-chbnw\" (UID: \"36349201-affc-4d02-9527-4f43167cdb1e\") " pod="openshift-cluster-node-tuning-operator/tuned-chbnw"
Jul 17 04:50:14.813111 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.812483    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/01097a42-523b-4764-b6a6-bf531915f32f-system-cni-dir\") pod \"multus-additional-cni-plugins-644hn\" (UID: \"01097a42-523b-4764-b6a6-bf531915f32f\") " pod="openshift-multus/multus-additional-cni-plugins-644hn"
Jul 17 04:50:14.813111 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.812486    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f4c22119-9902-4e1f-bc76-97d51253b16c-multus-daemon-config\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.813111 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:14.812535    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 04:50:14.813111 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.812537    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f4c22119-9902-4e1f-bc76-97d51253b16c-os-release\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.813111 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.812549    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/01097a42-523b-4764-b6a6-bf531915f32f-system-cni-dir\") pod \"multus-additional-cni-plugins-644hn\" (UID: \"01097a42-523b-4764-b6a6-bf531915f32f\") " pod="openshift-multus/multus-additional-cni-plugins-644hn"
Jul 17 04:50:14.813111 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:14.812618    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a54b09be-6889-45d6-a748-68e5b94b73d2-metrics-certs podName:a54b09be-6889-45d6-a748-68e5b94b73d2 nodeName:}" failed. No retries permitted until 2026-07-17 04:50:15.312576793 +0000 UTC m=+2.996170435 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a54b09be-6889-45d6-a748-68e5b94b73d2-metrics-certs") pod "network-metrics-daemon-rzdct" (UID: "a54b09be-6889-45d6-a748-68e5b94b73d2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 04:50:14.813111 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.812627    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f4c22119-9902-4e1f-bc76-97d51253b16c-multus-conf-dir\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.813111 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.812642    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/01097a42-523b-4764-b6a6-bf531915f32f-os-release\") pod \"multus-additional-cni-plugins-644hn\" (UID: \"01097a42-523b-4764-b6a6-bf531915f32f\") " pod="openshift-multus/multus-additional-cni-plugins-644hn"
Jul 17 04:50:14.813111 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.812669    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/36349201-affc-4d02-9527-4f43167cdb1e-lib-modules\") pod \"tuned-chbnw\" (UID: \"36349201-affc-4d02-9527-4f43167cdb1e\") " pod="openshift-cluster-node-tuning-operator/tuned-chbnw"
Jul 17 04:50:14.813111 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.812704    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/36349201-affc-4d02-9527-4f43167cdb1e-host\") pod \"tuned-chbnw\" (UID: \"36349201-affc-4d02-9527-4f43167cdb1e\") " pod="openshift-cluster-node-tuning-operator/tuned-chbnw"
Jul 17 04:50:14.813111 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.812728    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f4c22119-9902-4e1f-bc76-97d51253b16c-multus-socket-dir-parent\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.813111 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.812750    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f4c22119-9902-4e1f-bc76-97d51253b16c-host-run-k8s-cni-cncf-io\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.813111 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.812798    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/14db56e5-42df-4de1-bb13-86d893eed9c0-host-slash\") pod \"iptables-alerter-kfkjz\" (UID: \"14db56e5-42df-4de1-bb13-86d893eed9c0\") " pod="openshift-network-operator/iptables-alerter-kfkjz"
Jul 17 04:50:14.813911 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.812810    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f4c22119-9902-4e1f-bc76-97d51253b16c-multus-socket-dir-parent\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.813911 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.812817    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqgfw\" (UniqueName: \"kubernetes.io/projected/14db56e5-42df-4de1-bb13-86d893eed9c0-kube-api-access-jqgfw\") pod \"iptables-alerter-kfkjz\" (UID: \"14db56e5-42df-4de1-bb13-86d893eed9c0\") " pod="openshift-network-operator/iptables-alerter-kfkjz"
Jul 17 04:50:14.813911 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.812832    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/36349201-affc-4d02-9527-4f43167cdb1e-etc-sysconfig\") pod \"tuned-chbnw\" (UID: \"36349201-affc-4d02-9527-4f43167cdb1e\") " pod="openshift-cluster-node-tuning-operator/tuned-chbnw"
Jul 17 04:50:14.813911 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.812847    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f4c22119-9902-4e1f-bc76-97d51253b16c-cni-binary-copy\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.813911 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.812888    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c1baeee2-350c-43c5-9a04-fb34378edb5c-var-lib-openvswitch\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.813911 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.812906    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/237abccc-c19c-4fca-be94-d3013f3c2e86-dbus\") pod \"global-pull-secret-syncer-6bkt5\" (UID: \"237abccc-c19c-4fca-be94-d3013f3c2e86\") " pod="kube-system/global-pull-secret-syncer-6bkt5"
Jul 17 04:50:14.813911 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.812916    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/43b05be8-ccbf-412d-ad62-74596e1710fa-serviceca\") pod \"node-ca-bwdpn\" (UID: \"43b05be8-ccbf-412d-ad62-74596e1710fa\") " pod="openshift-image-registry/node-ca-bwdpn"
Jul 17 04:50:14.813911 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.812934    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gsgv\" (UniqueName: \"kubernetes.io/projected/01097a42-523b-4764-b6a6-bf531915f32f-kube-api-access-5gsgv\") pod \"multus-additional-cni-plugins-644hn\" (UID: \"01097a42-523b-4764-b6a6-bf531915f32f\") " pod="openshift-multus/multus-additional-cni-plugins-644hn"
Jul 17 04:50:14.813911 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.812948    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/01097a42-523b-4764-b6a6-bf531915f32f-os-release\") pod \"multus-additional-cni-plugins-644hn\" (UID: \"01097a42-523b-4764-b6a6-bf531915f32f\") " pod="openshift-multus/multus-additional-cni-plugins-644hn"
Jul 17 04:50:14.813911 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.812905    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/36349201-affc-4d02-9527-4f43167cdb1e-lib-modules\") pod \"tuned-chbnw\" (UID: \"36349201-affc-4d02-9527-4f43167cdb1e\") " pod="openshift-cluster-node-tuning-operator/tuned-chbnw"
Jul 17 04:50:14.813911 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.813081    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/da4d7ee5-31c8-465a-b5bd-5127ac635d08-konnectivity-ca\") pod \"konnectivity-agent-nbdgc\" (UID: \"da4d7ee5-31c8-465a-b5bd-5127ac635d08\") " pod="kube-system/konnectivity-agent-nbdgc"
Jul 17 04:50:14.813911 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.813107    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/36349201-affc-4d02-9527-4f43167cdb1e-host\") pod \"tuned-chbnw\" (UID: \"36349201-affc-4d02-9527-4f43167cdb1e\") " pod="openshift-cluster-node-tuning-operator/tuned-chbnw"
Jul 17 04:50:14.813911 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.813109    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/36349201-affc-4d02-9527-4f43167cdb1e-etc-tuned\") pod \"tuned-chbnw\" (UID: \"36349201-affc-4d02-9527-4f43167cdb1e\") " pod="openshift-cluster-node-tuning-operator/tuned-chbnw"
Jul 17 04:50:14.813911 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.813141    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/36349201-affc-4d02-9527-4f43167cdb1e-etc-sysconfig\") pod \"tuned-chbnw\" (UID: \"36349201-affc-4d02-9527-4f43167cdb1e\") " pod="openshift-cluster-node-tuning-operator/tuned-chbnw"
Jul 17 04:50:14.813911 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.813154    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f4c22119-9902-4e1f-bc76-97d51253b16c-host-var-lib-kubelet\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.813911 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.813186    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f4c22119-9902-4e1f-bc76-97d51253b16c-etc-kubernetes\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.813911 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.813163    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/14db56e5-42df-4de1-bb13-86d893eed9c0-host-slash\") pod \"iptables-alerter-kfkjz\" (UID: \"14db56e5-42df-4de1-bb13-86d893eed9c0\") " pod="openshift-network-operator/iptables-alerter-kfkjz"
Jul 17 04:50:14.814826 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.813252    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c1baeee2-350c-43c5-9a04-fb34378edb5c-host-run-netns\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.814826 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.813293    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c1baeee2-350c-43c5-9a04-fb34378edb5c-log-socket\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.814826 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.813318    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/01097a42-523b-4764-b6a6-bf531915f32f-cnibin\") pod \"multus-additional-cni-plugins-644hn\" (UID: \"01097a42-523b-4764-b6a6-bf531915f32f\") " pod="openshift-multus/multus-additional-cni-plugins-644hn"
Jul 17 04:50:14.814826 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.813342    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/da4d7ee5-31c8-465a-b5bd-5127ac635d08-agent-certs\") pod \"konnectivity-agent-nbdgc\" (UID: \"da4d7ee5-31c8-465a-b5bd-5127ac635d08\") " pod="kube-system/konnectivity-agent-nbdgc"
Jul 17 04:50:14.814826 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.813460    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7960477c-6164-45a9-a338-3063feb4a4a2-sys-fs\") pod \"aws-ebs-csi-driver-node-7tkkm\" (UID: \"7960477c-6164-45a9-a338-3063feb4a4a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7tkkm"
Jul 17 04:50:14.814826 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.813485    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c1baeee2-350c-43c5-9a04-fb34378edb5c-env-overrides\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.814826 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.813498    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f4c22119-9902-4e1f-bc76-97d51253b16c-cni-binary-copy\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.814826 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.813510    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/052c1550-58cb-4558-8b40-d9bd51a597b9-hosts-file\") pod \"node-resolver-4l4dn\" (UID: \"052c1550-58cb-4558-8b40-d9bd51a597b9\") " pod="openshift-dns/node-resolver-4l4dn"
Jul 17 04:50:14.814826 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.813538    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/01097a42-523b-4764-b6a6-bf531915f32f-cni-binary-copy\") pod \"multus-additional-cni-plugins-644hn\" (UID: \"01097a42-523b-4764-b6a6-bf531915f32f\") " pod="openshift-multus/multus-additional-cni-plugins-644hn"
Jul 17 04:50:14.814826 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.813562    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql6wg\" (UniqueName: \"kubernetes.io/projected/43b05be8-ccbf-412d-ad62-74596e1710fa-kube-api-access-ql6wg\") pod \"node-ca-bwdpn\" (UID: \"43b05be8-ccbf-412d-ad62-74596e1710fa\") " pod="openshift-image-registry/node-ca-bwdpn"
Jul 17 04:50:14.814826 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.813590    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/36349201-affc-4d02-9527-4f43167cdb1e-etc-systemd\") pod \"tuned-chbnw\" (UID: \"36349201-affc-4d02-9527-4f43167cdb1e\") " pod="openshift-cluster-node-tuning-operator/tuned-chbnw"
Jul 17 04:50:14.814826 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.813644    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbw2m\" (UniqueName: \"kubernetes.io/projected/36349201-affc-4d02-9527-4f43167cdb1e-kube-api-access-hbw2m\") pod \"tuned-chbnw\" (UID: \"36349201-affc-4d02-9527-4f43167cdb1e\") " pod="openshift-cluster-node-tuning-operator/tuned-chbnw"
Jul 17 04:50:14.814826 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.813668    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4c22119-9902-4e1f-bc76-97d51253b16c-multus-cni-dir\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.814826 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.813693    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f4c22119-9902-4e1f-bc76-97d51253b16c-host-var-lib-cni-multus\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.814826 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.813720    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/14db56e5-42df-4de1-bb13-86d893eed9c0-iptables-alerter-script\") pod \"iptables-alerter-kfkjz\" (UID: \"14db56e5-42df-4de1-bb13-86d893eed9c0\") " pod="openshift-network-operator/iptables-alerter-kfkjz"
Jul 17 04:50:14.814826 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.813745    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7fph\" (UniqueName: \"kubernetes.io/projected/7960477c-6164-45a9-a338-3063feb4a4a2-kube-api-access-x7fph\") pod \"aws-ebs-csi-driver-node-7tkkm\" (UID: \"7960477c-6164-45a9-a338-3063feb4a4a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7tkkm"
Jul 17 04:50:14.814826 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.813774    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/01097a42-523b-4764-b6a6-bf531915f32f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-644hn\" (UID: \"01097a42-523b-4764-b6a6-bf531915f32f\") " pod="openshift-multus/multus-additional-cni-plugins-644hn"
Jul 17 04:50:14.815566 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.813799    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/36349201-affc-4d02-9527-4f43167cdb1e-etc-kubernetes\") pod \"tuned-chbnw\" (UID: \"36349201-affc-4d02-9527-4f43167cdb1e\") " pod="openshift-cluster-node-tuning-operator/tuned-chbnw"
Jul 17 04:50:14.815566 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.813803    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/da4d7ee5-31c8-465a-b5bd-5127ac635d08-konnectivity-ca\") pod \"konnectivity-agent-nbdgc\" (UID: \"da4d7ee5-31c8-465a-b5bd-5127ac635d08\") " pod="kube-system/konnectivity-agent-nbdgc"
Jul 17 04:50:14.815566 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.813869    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/36349201-affc-4d02-9527-4f43167cdb1e-etc-kubernetes\") pod \"tuned-chbnw\" (UID: \"36349201-affc-4d02-9527-4f43167cdb1e\") " pod="openshift-cluster-node-tuning-operator/tuned-chbnw"
Jul 17 04:50:14.815566 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.813918    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f4c22119-9902-4e1f-bc76-97d51253b16c-host-run-k8s-cni-cncf-io\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.815566 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.813979    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/36349201-affc-4d02-9527-4f43167cdb1e-etc-systemd\") pod \"tuned-chbnw\" (UID: \"36349201-affc-4d02-9527-4f43167cdb1e\") " pod="openshift-cluster-node-tuning-operator/tuned-chbnw"
Jul 17 04:50:14.815566 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.814013    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f4c22119-9902-4e1f-bc76-97d51253b16c-etc-kubernetes\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.815566 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.814100    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bmqm\" (UniqueName: \"kubernetes.io/projected/a54b09be-6889-45d6-a748-68e5b94b73d2-kube-api-access-4bmqm\") pod \"network-metrics-daemon-rzdct\" (UID: \"a54b09be-6889-45d6-a748-68e5b94b73d2\") " pod="openshift-multus/network-metrics-daemon-rzdct"
Jul 17 04:50:14.815566 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.814149    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4c22119-9902-4e1f-bc76-97d51253b16c-multus-cni-dir\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.815566 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.814197    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/01097a42-523b-4764-b6a6-bf531915f32f-cnibin\") pod \"multus-additional-cni-plugins-644hn\" (UID: \"01097a42-523b-4764-b6a6-bf531915f32f\") " pod="openshift-multus/multus-additional-cni-plugins-644hn"
Jul 17 04:50:14.815566 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.814234    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c1baeee2-350c-43c5-9a04-fb34378edb5c-host-cni-bin\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.815566 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.814237    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f4c22119-9902-4e1f-bc76-97d51253b16c-host-var-lib-kubelet\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.815566 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.814264    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/052c1550-58cb-4558-8b40-d9bd51a597b9-tmp-dir\") pod \"node-resolver-4l4dn\" (UID: \"052c1550-58cb-4558-8b40-d9bd51a597b9\") " pod="openshift-dns/node-resolver-4l4dn"
Jul 17 04:50:14.815566 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.814383    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4c22119-9902-4e1f-bc76-97d51253b16c-system-cni-dir\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.815566 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.814409    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/01097a42-523b-4764-b6a6-bf531915f32f-cni-binary-copy\") pod \"multus-additional-cni-plugins-644hn\" (UID: \"01097a42-523b-4764-b6a6-bf531915f32f\") " pod="openshift-multus/multus-additional-cni-plugins-644hn"
Jul 17 04:50:14.815566 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.814417    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f4c22119-9902-4e1f-bc76-97d51253b16c-host-var-lib-cni-multus\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.815566 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.814469    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4c22119-9902-4e1f-bc76-97d51253b16c-system-cni-dir\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.815566 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.814495    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f4c22119-9902-4e1f-bc76-97d51253b16c-host-run-netns\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.815566 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.814532    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7960477c-6164-45a9-a338-3063feb4a4a2-registration-dir\") pod \"aws-ebs-csi-driver-node-7tkkm\" (UID: \"7960477c-6164-45a9-a338-3063feb4a4a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7tkkm"
Jul 17 04:50:14.816369 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.814561    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7960477c-6164-45a9-a338-3063feb4a4a2-etc-selinux\") pod \"aws-ebs-csi-driver-node-7tkkm\" (UID: \"7960477c-6164-45a9-a338-3063feb4a4a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7tkkm"
Jul 17 04:50:14.816369 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.814575    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f4c22119-9902-4e1f-bc76-97d51253b16c-host-run-netns\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.816369 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.814639    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c1baeee2-350c-43c5-9a04-fb34378edb5c-systemd-units\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.816369 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.814669    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/36349201-affc-4d02-9527-4f43167cdb1e-etc-modprobe-d\") pod \"tuned-chbnw\" (UID: \"36349201-affc-4d02-9527-4f43167cdb1e\") " pod="openshift-cluster-node-tuning-operator/tuned-chbnw"
Jul 17 04:50:14.816369 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.814705    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f4c22119-9902-4e1f-bc76-97d51253b16c-cnibin\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.816369 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.814699    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/01097a42-523b-4764-b6a6-bf531915f32f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-644hn\" (UID: \"01097a42-523b-4764-b6a6-bf531915f32f\") " pod="openshift-multus/multus-additional-cni-plugins-644hn"
Jul 17 04:50:14.816369 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.814756    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4xwm\" (UniqueName: \"kubernetes.io/projected/bcee811d-1474-45f6-b0c7-afceee510ccd-kube-api-access-l4xwm\") pod \"network-check-target-9vmqd\" (UID: \"bcee811d-1474-45f6-b0c7-afceee510ccd\") " pod="openshift-network-diagnostics/network-check-target-9vmqd"
Jul 17 04:50:14.816369 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.814758    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/36349201-affc-4d02-9527-4f43167cdb1e-etc-modprobe-d\") pod \"tuned-chbnw\" (UID: \"36349201-affc-4d02-9527-4f43167cdb1e\") " pod="openshift-cluster-node-tuning-operator/tuned-chbnw"
Jul 17 04:50:14.816369 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.814797    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/14db56e5-42df-4de1-bb13-86d893eed9c0-iptables-alerter-script\") pod \"iptables-alerter-kfkjz\" (UID: \"14db56e5-42df-4de1-bb13-86d893eed9c0\") " pod="openshift-network-operator/iptables-alerter-kfkjz"
Jul 17 04:50:14.816369 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.814802    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/237abccc-c19c-4fca-be94-d3013f3c2e86-kubelet-config\") pod \"global-pull-secret-syncer-6bkt5\" (UID: \"237abccc-c19c-4fca-be94-d3013f3c2e86\") " pod="kube-system/global-pull-secret-syncer-6bkt5"
Jul 17 04:50:14.816369 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.814818    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f4c22119-9902-4e1f-bc76-97d51253b16c-cnibin\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.816369 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.815061    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/36349201-affc-4d02-9527-4f43167cdb1e-tmp\") pod \"tuned-chbnw\" (UID: \"36349201-affc-4d02-9527-4f43167cdb1e\") " pod="openshift-cluster-node-tuning-operator/tuned-chbnw"
Jul 17 04:50:14.817055 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.817033    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/da4d7ee5-31c8-465a-b5bd-5127ac635d08-agent-certs\") pod \"konnectivity-agent-nbdgc\" (UID: \"da4d7ee5-31c8-465a-b5bd-5127ac635d08\") " pod="kube-system/konnectivity-agent-nbdgc"
Jul 17 04:50:14.818985 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.818948    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/36349201-affc-4d02-9527-4f43167cdb1e-etc-tuned\") pod \"tuned-chbnw\" (UID: \"36349201-affc-4d02-9527-4f43167cdb1e\") " pod="openshift-cluster-node-tuning-operator/tuned-chbnw"
Jul 17 04:50:14.821018 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.820586    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-459x5\" (UniqueName: \"kubernetes.io/projected/f4c22119-9902-4e1f-bc76-97d51253b16c-kube-api-access-459x5\") pod \"multus-lbjqq\" (UID: \"f4c22119-9902-4e1f-bc76-97d51253b16c\") " pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:14.821107 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:14.821015    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 04:50:14.821107 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:14.821036    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 04:50:14.821107 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:14.821051    2562 projected.go:196] Error preparing data for projected volume kube-api-access-l4xwm for pod openshift-network-diagnostics/network-check-target-9vmqd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 04:50:14.821256 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:14.821114    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bcee811d-1474-45f6-b0c7-afceee510ccd-kube-api-access-l4xwm podName:bcee811d-1474-45f6-b0c7-afceee510ccd nodeName:}" failed. No retries permitted until 2026-07-17 04:50:15.321097282 +0000 UTC m=+3.004690920 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-l4xwm" (UniqueName: "kubernetes.io/projected/bcee811d-1474-45f6-b0c7-afceee510ccd-kube-api-access-l4xwm") pod "network-check-target-9vmqd" (UID: "bcee811d-1474-45f6-b0c7-afceee510ccd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 04:50:14.824492 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.824096    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bmqm\" (UniqueName: \"kubernetes.io/projected/a54b09be-6889-45d6-a748-68e5b94b73d2-kube-api-access-4bmqm\") pod \"network-metrics-daemon-rzdct\" (UID: \"a54b09be-6889-45d6-a748-68e5b94b73d2\") " pod="openshift-multus/network-metrics-daemon-rzdct"
Jul 17 04:50:14.824492 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.824235    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqgfw\" (UniqueName: \"kubernetes.io/projected/14db56e5-42df-4de1-bb13-86d893eed9c0-kube-api-access-jqgfw\") pod \"iptables-alerter-kfkjz\" (UID: \"14db56e5-42df-4de1-bb13-86d893eed9c0\") " pod="openshift-network-operator/iptables-alerter-kfkjz"
Jul 17 04:50:14.824492 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.824457    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbw2m\" (UniqueName: \"kubernetes.io/projected/36349201-affc-4d02-9527-4f43167cdb1e-kube-api-access-hbw2m\") pod \"tuned-chbnw\" (UID: \"36349201-affc-4d02-9527-4f43167cdb1e\") " pod="openshift-cluster-node-tuning-operator/tuned-chbnw"
Jul 17 04:50:14.825835 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.825810    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gsgv\" (UniqueName: \"kubernetes.io/projected/01097a42-523b-4764-b6a6-bf531915f32f-kube-api-access-5gsgv\") pod \"multus-additional-cni-plugins-644hn\" (UID: \"01097a42-523b-4764-b6a6-bf531915f32f\") " pod="openshift-multus/multus-additional-cni-plugins-644hn"
Jul 17 04:50:14.827737 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.827698    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql6wg\" (UniqueName: \"kubernetes.io/projected/43b05be8-ccbf-412d-ad62-74596e1710fa-kube-api-access-ql6wg\") pod \"node-ca-bwdpn\" (UID: \"43b05be8-ccbf-412d-ad62-74596e1710fa\") " pod="openshift-image-registry/node-ca-bwdpn"
Jul 17 04:50:14.884147 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.884092    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-19.ec2.internal" event={"ID":"aea730e39f6c59e91b6798ae3c762419","Type":"ContainerStarted","Data":"f068137c6abfd736c2da150a771a91bf27243f8c01d0dc94e2b50e76b826c26e"}
Jul 17 04:50:14.886801 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.886773    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-19.ec2.internal" event={"ID":"02dd81e7d773510fb8d800d8518d7fdd","Type":"ContainerStarted","Data":"e85c77ca4c43e27bad4ea7809e46f4486005bb7628bac64b1a501b0526246ddc"}
Jul 17 04:50:14.915173 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915139    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c1baeee2-350c-43c5-9a04-fb34378edb5c-var-lib-openvswitch\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.915315 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915183    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/237abccc-c19c-4fca-be94-d3013f3c2e86-dbus\") pod \"global-pull-secret-syncer-6bkt5\" (UID: \"237abccc-c19c-4fca-be94-d3013f3c2e86\") " pod="kube-system/global-pull-secret-syncer-6bkt5"
Jul 17 04:50:14.915315 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915205    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c1baeee2-350c-43c5-9a04-fb34378edb5c-host-run-netns\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.915315 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915227    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c1baeee2-350c-43c5-9a04-fb34378edb5c-log-socket\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.915315 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915249    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7960477c-6164-45a9-a338-3063feb4a4a2-sys-fs\") pod \"aws-ebs-csi-driver-node-7tkkm\" (UID: \"7960477c-6164-45a9-a338-3063feb4a4a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7tkkm"
Jul 17 04:50:14.915315 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915256    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c1baeee2-350c-43c5-9a04-fb34378edb5c-var-lib-openvswitch\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.915315 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915270    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c1baeee2-350c-43c5-9a04-fb34378edb5c-env-overrides\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.915315 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915289    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/052c1550-58cb-4558-8b40-d9bd51a597b9-hosts-file\") pod \"node-resolver-4l4dn\" (UID: \"052c1550-58cb-4558-8b40-d9bd51a597b9\") " pod="openshift-dns/node-resolver-4l4dn"
Jul 17 04:50:14.915315 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915313    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7fph\" (UniqueName: \"kubernetes.io/projected/7960477c-6164-45a9-a338-3063feb4a4a2-kube-api-access-x7fph\") pod \"aws-ebs-csi-driver-node-7tkkm\" (UID: \"7960477c-6164-45a9-a338-3063feb4a4a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7tkkm"
Jul 17 04:50:14.915564 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915336    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7960477c-6164-45a9-a338-3063feb4a4a2-sys-fs\") pod \"aws-ebs-csi-driver-node-7tkkm\" (UID: \"7960477c-6164-45a9-a338-3063feb4a4a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7tkkm"
Jul 17 04:50:14.915564 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915338    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c1baeee2-350c-43c5-9a04-fb34378edb5c-host-cni-bin\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.915564 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915347    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/237abccc-c19c-4fca-be94-d3013f3c2e86-dbus\") pod \"global-pull-secret-syncer-6bkt5\" (UID: \"237abccc-c19c-4fca-be94-d3013f3c2e86\") " pod="kube-system/global-pull-secret-syncer-6bkt5"
Jul 17 04:50:14.915564 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915335    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c1baeee2-350c-43c5-9a04-fb34378edb5c-log-socket\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.915564 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915376    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c1baeee2-350c-43c5-9a04-fb34378edb5c-host-cni-bin\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.915564 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915375    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c1baeee2-350c-43c5-9a04-fb34378edb5c-host-run-netns\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.915564 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915469    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/052c1550-58cb-4558-8b40-d9bd51a597b9-hosts-file\") pod \"node-resolver-4l4dn\" (UID: \"052c1550-58cb-4558-8b40-d9bd51a597b9\") " pod="openshift-dns/node-resolver-4l4dn"
Jul 17 04:50:14.915564 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915378    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/052c1550-58cb-4558-8b40-d9bd51a597b9-tmp-dir\") pod \"node-resolver-4l4dn\" (UID: \"052c1550-58cb-4558-8b40-d9bd51a597b9\") " pod="openshift-dns/node-resolver-4l4dn"
Jul 17 04:50:14.915564 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915507    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7960477c-6164-45a9-a338-3063feb4a4a2-registration-dir\") pod \"aws-ebs-csi-driver-node-7tkkm\" (UID: \"7960477c-6164-45a9-a338-3063feb4a4a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7tkkm"
Jul 17 04:50:14.915564 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915530    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7960477c-6164-45a9-a338-3063feb4a4a2-etc-selinux\") pod \"aws-ebs-csi-driver-node-7tkkm\" (UID: \"7960477c-6164-45a9-a338-3063feb4a4a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7tkkm"
Jul 17 04:50:14.915564 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915553    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c1baeee2-350c-43c5-9a04-fb34378edb5c-systemd-units\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.915872 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915589    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/237abccc-c19c-4fca-be94-d3013f3c2e86-kubelet-config\") pod \"global-pull-secret-syncer-6bkt5\" (UID: \"237abccc-c19c-4fca-be94-d3013f3c2e86\") " pod="kube-system/global-pull-secret-syncer-6bkt5"
Jul 17 04:50:14.915872 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915615    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c1baeee2-350c-43c5-9a04-fb34378edb5c-host-kubelet\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.915872 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915634    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c1baeee2-350c-43c5-9a04-fb34378edb5c-run-systemd\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.915872 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915655    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c1baeee2-350c-43c5-9a04-fb34378edb5c-node-log\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.915872 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915665    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7960477c-6164-45a9-a338-3063feb4a4a2-etc-selinux\") pod \"aws-ebs-csi-driver-node-7tkkm\" (UID: \"7960477c-6164-45a9-a338-3063feb4a4a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7tkkm"
Jul 17 04:50:14.915872 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915683    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/052c1550-58cb-4558-8b40-d9bd51a597b9-tmp-dir\") pod \"node-resolver-4l4dn\" (UID: \"052c1550-58cb-4558-8b40-d9bd51a597b9\") " pod="openshift-dns/node-resolver-4l4dn"
Jul 17 04:50:14.915872 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915683    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7960477c-6164-45a9-a338-3063feb4a4a2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7tkkm\" (UID: \"7960477c-6164-45a9-a338-3063feb4a4a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7tkkm"
Jul 17 04:50:14.915872 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915716    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7960477c-6164-45a9-a338-3063feb4a4a2-registration-dir\") pod \"aws-ebs-csi-driver-node-7tkkm\" (UID: \"7960477c-6164-45a9-a338-3063feb4a4a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7tkkm"
Jul 17 04:50:14.915872 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915728    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c1baeee2-350c-43c5-9a04-fb34378edb5c-ovnkube-config\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.915872 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915747    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c1baeee2-350c-43c5-9a04-fb34378edb5c-systemd-units\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.915872 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915763    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/237abccc-c19c-4fca-be94-d3013f3c2e86-original-pull-secret\") pod \"global-pull-secret-syncer-6bkt5\" (UID: \"237abccc-c19c-4fca-be94-d3013f3c2e86\") " pod="kube-system/global-pull-secret-syncer-6bkt5"
Jul 17 04:50:14.915872 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915786    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7960477c-6164-45a9-a338-3063feb4a4a2-socket-dir\") pod \"aws-ebs-csi-driver-node-7tkkm\" (UID: \"7960477c-6164-45a9-a338-3063feb4a4a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7tkkm"
Jul 17 04:50:14.915872 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915792    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c1baeee2-350c-43c5-9a04-fb34378edb5c-host-kubelet\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.915872 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915797    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7960477c-6164-45a9-a338-3063feb4a4a2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7tkkm\" (UID: \"7960477c-6164-45a9-a338-3063feb4a4a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7tkkm"
Jul 17 04:50:14.915872 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915814    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c1baeee2-350c-43c5-9a04-fb34378edb5c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.915872 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915834    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c1baeee2-350c-43c5-9a04-fb34378edb5c-run-systemd\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.915872 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915838    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c1baeee2-350c-43c5-9a04-fb34378edb5c-node-log\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.916430 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915764    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/237abccc-c19c-4fca-be94-d3013f3c2e86-kubelet-config\") pod \"global-pull-secret-syncer-6bkt5\" (UID: \"237abccc-c19c-4fca-be94-d3013f3c2e86\") " pod="kube-system/global-pull-secret-syncer-6bkt5"
Jul 17 04:50:14.916430 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915850    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c1baeee2-350c-43c5-9a04-fb34378edb5c-env-overrides\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.916430 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915841    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c1baeee2-350c-43c5-9a04-fb34378edb5c-ovn-node-metrics-cert\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.916430 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915900    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jt92\" (UniqueName: \"kubernetes.io/projected/c1baeee2-350c-43c5-9a04-fb34378edb5c-kube-api-access-4jt92\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.916430 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915919    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8ptp\" (UniqueName: \"kubernetes.io/projected/052c1550-58cb-4558-8b40-d9bd51a597b9-kube-api-access-r8ptp\") pod \"node-resolver-4l4dn\" (UID: \"052c1550-58cb-4558-8b40-d9bd51a597b9\") " pod="openshift-dns/node-resolver-4l4dn"
Jul 17 04:50:14.916430 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915926    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7960477c-6164-45a9-a338-3063feb4a4a2-socket-dir\") pod \"aws-ebs-csi-driver-node-7tkkm\" (UID: \"7960477c-6164-45a9-a338-3063feb4a4a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7tkkm"
Jul 17 04:50:14.916430 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915936    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c1baeee2-350c-43c5-9a04-fb34378edb5c-etc-openvswitch\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.916430 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915899    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c1baeee2-350c-43c5-9a04-fb34378edb5c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.916430 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:14.915988    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 04:50:14.916430 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.915995    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c1baeee2-350c-43c5-9a04-fb34378edb5c-etc-openvswitch\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.916430 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:14.916044    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/237abccc-c19c-4fca-be94-d3013f3c2e86-original-pull-secret podName:237abccc-c19c-4fca-be94-d3013f3c2e86 nodeName:}" failed. No retries permitted until 2026-07-17 04:50:15.416026302 +0000 UTC m=+3.099619961 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/237abccc-c19c-4fca-be94-d3013f3c2e86-original-pull-secret") pod "global-pull-secret-syncer-6bkt5" (UID: "237abccc-c19c-4fca-be94-d3013f3c2e86") : object "kube-system"/"original-pull-secret" not registered
Jul 17 04:50:14.916430 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.916123    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c1baeee2-350c-43c5-9a04-fb34378edb5c-host-run-ovn-kubernetes\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.916430 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.916143    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c1baeee2-350c-43c5-9a04-fb34378edb5c-host-slash\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.916430 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.916168    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7960477c-6164-45a9-a338-3063feb4a4a2-device-dir\") pod \"aws-ebs-csi-driver-node-7tkkm\" (UID: \"7960477c-6164-45a9-a338-3063feb4a4a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7tkkm"
Jul 17 04:50:14.916430 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.916171    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c1baeee2-350c-43c5-9a04-fb34378edb5c-host-run-ovn-kubernetes\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.916430 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.916183    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c1baeee2-350c-43c5-9a04-fb34378edb5c-run-ovn\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.916430 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.916210    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c1baeee2-350c-43c5-9a04-fb34378edb5c-run-ovn\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.917121 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.916194    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c1baeee2-350c-43c5-9a04-fb34378edb5c-host-slash\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.917121 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.916229    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7960477c-6164-45a9-a338-3063feb4a4a2-device-dir\") pod \"aws-ebs-csi-driver-node-7tkkm\" (UID: \"7960477c-6164-45a9-a338-3063feb4a4a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7tkkm"
Jul 17 04:50:14.917121 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.916258    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c1baeee2-350c-43c5-9a04-fb34378edb5c-host-cni-netd\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.917121 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.916284    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c1baeee2-350c-43c5-9a04-fb34378edb5c-run-openvswitch\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.917121 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.916309    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c1baeee2-350c-43c5-9a04-fb34378edb5c-ovnkube-script-lib\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.917121 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.916336    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c1baeee2-350c-43c5-9a04-fb34378edb5c-ovnkube-config\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.917121 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.916338    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c1baeee2-350c-43c5-9a04-fb34378edb5c-host-cni-netd\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.917121 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.916337    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c1baeee2-350c-43c5-9a04-fb34378edb5c-run-openvswitch\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.917121 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.916725    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c1baeee2-350c-43c5-9a04-fb34378edb5c-ovnkube-script-lib\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.918990 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.918691    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c1baeee2-350c-43c5-9a04-fb34378edb5c-ovn-node-metrics-cert\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.925227 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.925204    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jt92\" (UniqueName: \"kubernetes.io/projected/c1baeee2-350c-43c5-9a04-fb34378edb5c-kube-api-access-4jt92\") pod \"ovnkube-node-tx9gj\" (UID: \"c1baeee2-350c-43c5-9a04-fb34378edb5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:14.925329 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.925228    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8ptp\" (UniqueName: \"kubernetes.io/projected/052c1550-58cb-4558-8b40-d9bd51a597b9-kube-api-access-r8ptp\") pod \"node-resolver-4l4dn\" (UID: \"052c1550-58cb-4558-8b40-d9bd51a597b9\") " pod="openshift-dns/node-resolver-4l4dn"
Jul 17 04:50:14.926596 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.926574    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7fph\" (UniqueName: \"kubernetes.io/projected/7960477c-6164-45a9-a338-3063feb4a4a2-kube-api-access-x7fph\") pod \"aws-ebs-csi-driver-node-7tkkm\" (UID: \"7960477c-6164-45a9-a338-3063feb4a4a2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7tkkm"
Jul 17 04:50:14.949821 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:14.949743    2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 04:50:15.002245 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:15.002210    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-644hn"
Jul 17 04:50:15.012031 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:15.012006    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lbjqq"
Jul 17 04:50:15.024945 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:15.024926    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kfkjz"
Jul 17 04:50:15.029780 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:15.029759    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bwdpn"
Jul 17 04:50:15.036888 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:15.036865    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nbdgc"
Jul 17 04:50:15.046827 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:15.046803    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-chbnw"
Jul 17 04:50:15.054832 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:15.054812    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:15.065945 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:15.065922    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7tkkm"
Jul 17 04:50:15.070167 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:15.070144    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4l4dn"
Jul 17 04:50:15.322597 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:15.318942    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a54b09be-6889-45d6-a748-68e5b94b73d2-metrics-certs\") pod \"network-metrics-daemon-rzdct\" (UID: \"a54b09be-6889-45d6-a748-68e5b94b73d2\") " pod="openshift-multus/network-metrics-daemon-rzdct"
Jul 17 04:50:15.322597 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:15.319081    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 04:50:15.322597 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:15.319147    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a54b09be-6889-45d6-a748-68e5b94b73d2-metrics-certs podName:a54b09be-6889-45d6-a748-68e5b94b73d2 nodeName:}" failed. No retries permitted until 2026-07-17 04:50:16.319128523 +0000 UTC m=+4.002722158 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a54b09be-6889-45d6-a748-68e5b94b73d2-metrics-certs") pod "network-metrics-daemon-rzdct" (UID: "a54b09be-6889-45d6-a748-68e5b94b73d2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 04:50:15.420190 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:15.420159    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4xwm\" (UniqueName: \"kubernetes.io/projected/bcee811d-1474-45f6-b0c7-afceee510ccd-kube-api-access-l4xwm\") pod \"network-check-target-9vmqd\" (UID: \"bcee811d-1474-45f6-b0c7-afceee510ccd\") " pod="openshift-network-diagnostics/network-check-target-9vmqd"
Jul 17 04:50:15.420190 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:15.420200    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/237abccc-c19c-4fca-be94-d3013f3c2e86-original-pull-secret\") pod \"global-pull-secret-syncer-6bkt5\" (UID: \"237abccc-c19c-4fca-be94-d3013f3c2e86\") " pod="kube-system/global-pull-secret-syncer-6bkt5"
Jul 17 04:50:15.420438 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:15.420307    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 04:50:15.420438 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:15.420325    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 04:50:15.420438 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:15.420341    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 04:50:15.420438 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:15.420354    2562 projected.go:196] Error preparing data for projected volume kube-api-access-l4xwm for pod openshift-network-diagnostics/network-check-target-9vmqd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 04:50:15.420438 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:15.420372    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/237abccc-c19c-4fca-be94-d3013f3c2e86-original-pull-secret podName:237abccc-c19c-4fca-be94-d3013f3c2e86 nodeName:}" failed. No retries permitted until 2026-07-17 04:50:16.420354185 +0000 UTC m=+4.103947837 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/237abccc-c19c-4fca-be94-d3013f3c2e86-original-pull-secret") pod "global-pull-secret-syncer-6bkt5" (UID: "237abccc-c19c-4fca-be94-d3013f3c2e86") : object "kube-system"/"original-pull-secret" not registered
Jul 17 04:50:15.420438 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:15.420388    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bcee811d-1474-45f6-b0c7-afceee510ccd-kube-api-access-l4xwm podName:bcee811d-1474-45f6-b0c7-afceee510ccd nodeName:}" failed. No retries permitted until 2026-07-17 04:50:16.420379121 +0000 UTC m=+4.103972760 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-l4xwm" (UniqueName: "kubernetes.io/projected/bcee811d-1474-45f6-b0c7-afceee510ccd-kube-api-access-l4xwm") pod "network-check-target-9vmqd" (UID: "bcee811d-1474-45f6-b0c7-afceee510ccd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 04:50:15.739307 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:15.739262    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-16 04:45:14 +0000 UTC" deadline="2028-02-21 02:06:21.074769346 +0000 UTC"
Jul 17 04:50:15.739307 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:15.739316    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14013h16m5.335473039s"
Jul 17 04:50:15.878109 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:15.878067    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzdct"
Jul 17 04:50:15.878273 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:15.878209    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzdct" podUID="a54b09be-6889-45d6-a748-68e5b94b73d2"
Jul 17 04:50:15.992361 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:15.992335    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14db56e5_42df_4de1_bb13_86d893eed9c0.slice/crio-85dc13c9c7a5a84a656bd8a7e6b0c23c6121096cac825d0446a586dfcd4e3d5d WatchSource:0}: Error finding container 85dc13c9c7a5a84a656bd8a7e6b0c23c6121096cac825d0446a586dfcd4e3d5d: Status 404 returned error can't find the container with id 85dc13c9c7a5a84a656bd8a7e6b0c23c6121096cac825d0446a586dfcd4e3d5d
Jul 17 04:50:15.993514 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:15.993492    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01097a42_523b_4764_b6a6_bf531915f32f.slice/crio-9d2d77cc5daa8c3480f876681dd375f9b781a18c3b20af53b6e5a26fa2f3a7c2 WatchSource:0}: Error finding container 9d2d77cc5daa8c3480f876681dd375f9b781a18c3b20af53b6e5a26fa2f3a7c2: Status 404 returned error can't find the container with id 9d2d77cc5daa8c3480f876681dd375f9b781a18c3b20af53b6e5a26fa2f3a7c2
Jul 17 04:50:15.999773 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:15.999713    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43b05be8_ccbf_412d_ad62_74596e1710fa.slice/crio-b4076f6c7832fbe9feafc2a52cde69fa5f85c74ad7844e09ad8d6dfecab4c856 WatchSource:0}: Error finding container b4076f6c7832fbe9feafc2a52cde69fa5f85c74ad7844e09ad8d6dfecab4c856: Status 404 returned error can't find the container with id b4076f6c7832fbe9feafc2a52cde69fa5f85c74ad7844e09ad8d6dfecab4c856
Jul 17 04:50:15.999992 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:50:15.999946    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod052c1550_58cb_4558_8b40_d9bd51a597b9.slice/crio-171b197f6017ee0db2094e9ffdf3df9a8d8d0a2fd67a6b0e7d9c77e0437e6c14 WatchSource:0}: Error finding container 171b197f6017ee0db2094e9ffdf3df9a8d8d0a2fd67a6b0e7d9c77e0437e6c14: Status 404 returned error can't find the container with id 171b197f6017ee0db2094e9ffdf3df9a8d8d0a2fd67a6b0e7d9c77e0437e6c14
Jul 17 04:50:16.325172 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:16.324921    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a54b09be-6889-45d6-a748-68e5b94b73d2-metrics-certs\") pod \"network-metrics-daemon-rzdct\" (UID: \"a54b09be-6889-45d6-a748-68e5b94b73d2\") " pod="openshift-multus/network-metrics-daemon-rzdct"
Jul 17 04:50:16.325312 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:16.325065    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 04:50:16.325312 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:16.325275    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a54b09be-6889-45d6-a748-68e5b94b73d2-metrics-certs podName:a54b09be-6889-45d6-a748-68e5b94b73d2 nodeName:}" failed. No retries permitted until 2026-07-17 04:50:18.325257862 +0000 UTC m=+6.008851497 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a54b09be-6889-45d6-a748-68e5b94b73d2-metrics-certs") pod "network-metrics-daemon-rzdct" (UID: "a54b09be-6889-45d6-a748-68e5b94b73d2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 04:50:16.425574 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:16.425543    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/237abccc-c19c-4fca-be94-d3013f3c2e86-original-pull-secret\") pod \"global-pull-secret-syncer-6bkt5\" (UID: \"237abccc-c19c-4fca-be94-d3013f3c2e86\") " pod="kube-system/global-pull-secret-syncer-6bkt5"
Jul 17 04:50:16.425701 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:16.425597    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4xwm\" (UniqueName: \"kubernetes.io/projected/bcee811d-1474-45f6-b0c7-afceee510ccd-kube-api-access-l4xwm\") pod \"network-check-target-9vmqd\" (UID: \"bcee811d-1474-45f6-b0c7-afceee510ccd\") " pod="openshift-network-diagnostics/network-check-target-9vmqd"
Jul 17 04:50:16.425701 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:16.425683    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 04:50:16.425701 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:16.425688    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 04:50:16.425810 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:16.425704    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 04:50:16.425810 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:16.425713    2562 projected.go:196] Error preparing data for projected volume kube-api-access-l4xwm for pod openshift-network-diagnostics/network-check-target-9vmqd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 04:50:16.425810 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:16.425741    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/237abccc-c19c-4fca-be94-d3013f3c2e86-original-pull-secret podName:237abccc-c19c-4fca-be94-d3013f3c2e86 nodeName:}" failed. No retries permitted until 2026-07-17 04:50:18.425725212 +0000 UTC m=+6.109318851 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/237abccc-c19c-4fca-be94-d3013f3c2e86-original-pull-secret") pod "global-pull-secret-syncer-6bkt5" (UID: "237abccc-c19c-4fca-be94-d3013f3c2e86") : object "kube-system"/"original-pull-secret" not registered
Jul 17 04:50:16.425810 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:16.425757    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bcee811d-1474-45f6-b0c7-afceee510ccd-kube-api-access-l4xwm podName:bcee811d-1474-45f6-b0c7-afceee510ccd nodeName:}" failed. No retries permitted until 2026-07-17 04:50:18.425748821 +0000 UTC m=+6.109342456 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-l4xwm" (UniqueName: "kubernetes.io/projected/bcee811d-1474-45f6-b0c7-afceee510ccd-kube-api-access-l4xwm") pod "network-check-target-9vmqd" (UID: "bcee811d-1474-45f6-b0c7-afceee510ccd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 04:50:16.740092 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:16.739976    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-16 04:45:14 +0000 UTC" deadline="2028-03-06 23:28:47.29642489 +0000 UTC"
Jul 17 04:50:16.740092 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:16.740034    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14370h38m30.55639476s"
Jul 17 04:50:16.881042 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:16.880233    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6bkt5"
Jul 17 04:50:16.881042 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:16.880358    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6bkt5" podUID="237abccc-c19c-4fca-be94-d3013f3c2e86"
Jul 17 04:50:16.881042 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:16.880741    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vmqd"
Jul 17 04:50:16.881594 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:16.881551    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vmqd" podUID="bcee811d-1474-45f6-b0c7-afceee510ccd"
Jul 17 04:50:16.908983 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:16.904917    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-19.ec2.internal" event={"ID":"02dd81e7d773510fb8d800d8518d7fdd","Type":"ContainerStarted","Data":"58c06b0f0d5e901b7ebdf91246c6b56e16a93b5355b3e3a34d66e47792826811"}
Jul 17 04:50:16.915605 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:16.915531    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nbdgc" event={"ID":"da4d7ee5-31c8-465a-b5bd-5127ac635d08","Type":"ContainerStarted","Data":"7a49b19391b27471c12fe63eeaf1ef9e6fd2bd0b6a2bd0361310706b1672fca6"}
Jul 17 04:50:16.919432 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:16.919137    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-19.ec2.internal" podStartSLOduration=2.919119519 podStartE2EDuration="2.919119519s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 04:50:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 04:50:16.918694792 +0000 UTC m=+4.602288447" watchObservedRunningTime="2026-07-17 04:50:16.919119519 +0000 UTC m=+4.602713179"
Jul 17 04:50:16.919672 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:16.919616    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lbjqq" event={"ID":"f4c22119-9902-4e1f-bc76-97d51253b16c","Type":"ContainerStarted","Data":"1e0aba420a9ae01a5e400f0d91435045dfada926a78d084748f4dfea17a17627"}
Jul 17 04:50:16.929064 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:16.928997    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4l4dn" event={"ID":"052c1550-58cb-4558-8b40-d9bd51a597b9","Type":"ContainerStarted","Data":"171b197f6017ee0db2094e9ffdf3df9a8d8d0a2fd67a6b0e7d9c77e0437e6c14"}
Jul 17 04:50:16.936253 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:16.935935    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj" event={"ID":"c1baeee2-350c-43c5-9a04-fb34378edb5c","Type":"ContainerStarted","Data":"c83b34f72e466a3e30def7bb4a57f688e9034c86fb9a191b244ad5015f276c18"}
Jul 17 04:50:16.940604 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:16.940534    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bwdpn" event={"ID":"43b05be8-ccbf-412d-ad62-74596e1710fa","Type":"ContainerStarted","Data":"b4076f6c7832fbe9feafc2a52cde69fa5f85c74ad7844e09ad8d6dfecab4c856"}
Jul 17 04:50:16.949791 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:16.949722    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-chbnw" event={"ID":"36349201-affc-4d02-9527-4f43167cdb1e","Type":"ContainerStarted","Data":"6aa01d4ccaf7d382ed11bd6b89f88805418e85982a049c950e771ea469bf641b"}
Jul 17 04:50:16.951702 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:16.951678    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7tkkm" event={"ID":"7960477c-6164-45a9-a338-3063feb4a4a2","Type":"ContainerStarted","Data":"3a59db6d4b8ca9581528ed37f6dc07db8217f19784b17665af0fd8609d7e4bc6"}
Jul 17 04:50:16.959796 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:16.959771    2562 generic.go:356] "Generic (PLEG): container finished" podID="aea730e39f6c59e91b6798ae3c762419" containerID="1c2db88ac6c08930981bd337e7af6f8a1c2a93811e5fe5e1c02271aea028bce6" exitCode=0
Jul 17 04:50:16.959904 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:16.959856    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-19.ec2.internal" event={"ID":"aea730e39f6c59e91b6798ae3c762419","Type":"ContainerDied","Data":"1c2db88ac6c08930981bd337e7af6f8a1c2a93811e5fe5e1c02271aea028bce6"}
Jul 17 04:50:16.970414 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:16.970388    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-644hn" event={"ID":"01097a42-523b-4764-b6a6-bf531915f32f","Type":"ContainerStarted","Data":"9d2d77cc5daa8c3480f876681dd375f9b781a18c3b20af53b6e5a26fa2f3a7c2"}
Jul 17 04:50:16.982283 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:16.982255    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kfkjz" event={"ID":"14db56e5-42df-4de1-bb13-86d893eed9c0","Type":"ContainerStarted","Data":"85dc13c9c7a5a84a656bd8a7e6b0c23c6121096cac825d0446a586dfcd4e3d5d"}
Jul 17 04:50:17.879448 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:17.878913    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzdct"
Jul 17 04:50:17.879448 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:17.879070    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzdct" podUID="a54b09be-6889-45d6-a748-68e5b94b73d2"
Jul 17 04:50:18.002298 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:18.002246    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-19.ec2.internal" event={"ID":"aea730e39f6c59e91b6798ae3c762419","Type":"ContainerStarted","Data":"f2fb1f0c966d74deed7b11d741a08303a45a54eb8f47e702bd113dc99ae55b3c"}
Jul 17 04:50:18.344772 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:18.344724    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a54b09be-6889-45d6-a748-68e5b94b73d2-metrics-certs\") pod \"network-metrics-daemon-rzdct\" (UID: \"a54b09be-6889-45d6-a748-68e5b94b73d2\") " pod="openshift-multus/network-metrics-daemon-rzdct"
Jul 17 04:50:18.344995 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:18.344884    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 04:50:18.344995 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:18.344946    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a54b09be-6889-45d6-a748-68e5b94b73d2-metrics-certs podName:a54b09be-6889-45d6-a748-68e5b94b73d2 nodeName:}" failed. No retries permitted until 2026-07-17 04:50:22.34492759 +0000 UTC m=+10.028521238 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a54b09be-6889-45d6-a748-68e5b94b73d2-metrics-certs") pod "network-metrics-daemon-rzdct" (UID: "a54b09be-6889-45d6-a748-68e5b94b73d2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 04:50:18.445673 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:18.445636    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4xwm\" (UniqueName: \"kubernetes.io/projected/bcee811d-1474-45f6-b0c7-afceee510ccd-kube-api-access-l4xwm\") pod \"network-check-target-9vmqd\" (UID: \"bcee811d-1474-45f6-b0c7-afceee510ccd\") " pod="openshift-network-diagnostics/network-check-target-9vmqd"
Jul 17 04:50:18.445852 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:18.445702    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/237abccc-c19c-4fca-be94-d3013f3c2e86-original-pull-secret\") pod \"global-pull-secret-syncer-6bkt5\" (UID: \"237abccc-c19c-4fca-be94-d3013f3c2e86\") " pod="kube-system/global-pull-secret-syncer-6bkt5"
Jul 17 04:50:18.445852 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:18.445786    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 04:50:18.445852 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:18.445815    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 04:50:18.445852 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:18.445829    2562 projected.go:196] Error preparing data for projected volume kube-api-access-l4xwm for pod openshift-network-diagnostics/network-check-target-9vmqd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 04:50:18.445852 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:18.445832    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 04:50:18.446126 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:18.445899    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/237abccc-c19c-4fca-be94-d3013f3c2e86-original-pull-secret podName:237abccc-c19c-4fca-be94-d3013f3c2e86 nodeName:}" failed. No retries permitted until 2026-07-17 04:50:22.445880436 +0000 UTC m=+10.129474071 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/237abccc-c19c-4fca-be94-d3013f3c2e86-original-pull-secret") pod "global-pull-secret-syncer-6bkt5" (UID: "237abccc-c19c-4fca-be94-d3013f3c2e86") : object "kube-system"/"original-pull-secret" not registered
Jul 17 04:50:18.446126 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:18.445922    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bcee811d-1474-45f6-b0c7-afceee510ccd-kube-api-access-l4xwm podName:bcee811d-1474-45f6-b0c7-afceee510ccd nodeName:}" failed. No retries permitted until 2026-07-17 04:50:22.445912846 +0000 UTC m=+10.129506487 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-l4xwm" (UniqueName: "kubernetes.io/projected/bcee811d-1474-45f6-b0c7-afceee510ccd-kube-api-access-l4xwm") pod "network-check-target-9vmqd" (UID: "bcee811d-1474-45f6-b0c7-afceee510ccd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 04:50:18.880954 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:18.880919    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vmqd"
Jul 17 04:50:18.881429 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:18.881075    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vmqd" podUID="bcee811d-1474-45f6-b0c7-afceee510ccd"
Jul 17 04:50:18.881499 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:18.881448    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6bkt5"
Jul 17 04:50:18.881557 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:18.881533    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6bkt5" podUID="237abccc-c19c-4fca-be94-d3013f3c2e86"
Jul 17 04:50:19.877906 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:19.877868    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzdct"
Jul 17 04:50:19.878091 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:19.878021    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzdct" podUID="a54b09be-6889-45d6-a748-68e5b94b73d2"
Jul 17 04:50:20.877713 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:20.877680    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vmqd"
Jul 17 04:50:20.878182 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:20.877811    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vmqd" podUID="bcee811d-1474-45f6-b0c7-afceee510ccd"
Jul 17 04:50:20.878182 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:20.878121    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6bkt5"
Jul 17 04:50:20.878294 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:20.878248    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6bkt5" podUID="237abccc-c19c-4fca-be94-d3013f3c2e86"
Jul 17 04:50:21.877745 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:21.877708    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzdct"
Jul 17 04:50:21.878228 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:21.877858    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzdct" podUID="a54b09be-6889-45d6-a748-68e5b94b73d2"
Jul 17 04:50:22.376618 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:22.376575    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a54b09be-6889-45d6-a748-68e5b94b73d2-metrics-certs\") pod \"network-metrics-daemon-rzdct\" (UID: \"a54b09be-6889-45d6-a748-68e5b94b73d2\") " pod="openshift-multus/network-metrics-daemon-rzdct"
Jul 17 04:50:22.376791 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:22.376763    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 04:50:22.376856 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:22.376836    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a54b09be-6889-45d6-a748-68e5b94b73d2-metrics-certs podName:a54b09be-6889-45d6-a748-68e5b94b73d2 nodeName:}" failed. No retries permitted until 2026-07-17 04:50:30.376815927 +0000 UTC m=+18.060409565 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a54b09be-6889-45d6-a748-68e5b94b73d2-metrics-certs") pod "network-metrics-daemon-rzdct" (UID: "a54b09be-6889-45d6-a748-68e5b94b73d2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 04:50:22.477414 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:22.477375    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4xwm\" (UniqueName: \"kubernetes.io/projected/bcee811d-1474-45f6-b0c7-afceee510ccd-kube-api-access-l4xwm\") pod \"network-check-target-9vmqd\" (UID: \"bcee811d-1474-45f6-b0c7-afceee510ccd\") " pod="openshift-network-diagnostics/network-check-target-9vmqd"
Jul 17 04:50:22.477597 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:22.477433    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/237abccc-c19c-4fca-be94-d3013f3c2e86-original-pull-secret\") pod \"global-pull-secret-syncer-6bkt5\" (UID: \"237abccc-c19c-4fca-be94-d3013f3c2e86\") " pod="kube-system/global-pull-secret-syncer-6bkt5"
Jul 17 04:50:22.479153 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:22.479123    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 04:50:22.479153 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:22.479152    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 04:50:22.479317 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:22.479175    2562 projected.go:196] Error preparing data for projected volume kube-api-access-l4xwm for pod openshift-network-diagnostics/network-check-target-9vmqd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 04:50:22.479317 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:22.479208    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 04:50:22.479317 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:22.479243    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bcee811d-1474-45f6-b0c7-afceee510ccd-kube-api-access-l4xwm podName:bcee811d-1474-45f6-b0c7-afceee510ccd nodeName:}" failed. No retries permitted until 2026-07-17 04:50:30.479215515 +0000 UTC m=+18.162809167 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-l4xwm" (UniqueName: "kubernetes.io/projected/bcee811d-1474-45f6-b0c7-afceee510ccd-kube-api-access-l4xwm") pod "network-check-target-9vmqd" (UID: "bcee811d-1474-45f6-b0c7-afceee510ccd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 04:50:22.479317 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:22.479283    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/237abccc-c19c-4fca-be94-d3013f3c2e86-original-pull-secret podName:237abccc-c19c-4fca-be94-d3013f3c2e86 nodeName:}" failed. No retries permitted until 2026-07-17 04:50:30.479266038 +0000 UTC m=+18.162859676 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/237abccc-c19c-4fca-be94-d3013f3c2e86-original-pull-secret") pod "global-pull-secret-syncer-6bkt5" (UID: "237abccc-c19c-4fca-be94-d3013f3c2e86") : object "kube-system"/"original-pull-secret" not registered
Jul 17 04:50:22.880125 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:22.879571    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6bkt5"
Jul 17 04:50:22.880125 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:22.879682    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6bkt5" podUID="237abccc-c19c-4fca-be94-d3013f3c2e86"
Jul 17 04:50:22.880125 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:22.880015    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vmqd"
Jul 17 04:50:22.880125 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:22.880090    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vmqd" podUID="bcee811d-1474-45f6-b0c7-afceee510ccd"
Jul 17 04:50:23.878594 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:23.878453    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzdct"
Jul 17 04:50:23.878780 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:23.878617    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzdct" podUID="a54b09be-6889-45d6-a748-68e5b94b73d2"
Jul 17 04:50:24.878228 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:24.878193    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6bkt5"
Jul 17 04:50:24.878672 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:24.878322    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6bkt5" podUID="237abccc-c19c-4fca-be94-d3013f3c2e86"
Jul 17 04:50:24.878730 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:24.878707    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vmqd"
Jul 17 04:50:24.878821 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:24.878802    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vmqd" podUID="bcee811d-1474-45f6-b0c7-afceee510ccd"
Jul 17 04:50:25.877637 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:25.877602    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzdct"
Jul 17 04:50:25.877811 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:25.877721    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzdct" podUID="a54b09be-6889-45d6-a748-68e5b94b73d2"
Jul 17 04:50:26.878233 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:26.878196    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vmqd"
Jul 17 04:50:26.878678 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:26.878196    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6bkt5"
Jul 17 04:50:26.878678 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:26.878329    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vmqd" podUID="bcee811d-1474-45f6-b0c7-afceee510ccd"
Jul 17 04:50:26.878678 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:26.878386    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6bkt5" podUID="237abccc-c19c-4fca-be94-d3013f3c2e86"
Jul 17 04:50:27.877841 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:27.877812    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzdct"
Jul 17 04:50:27.878015 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:27.877956    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzdct" podUID="a54b09be-6889-45d6-a748-68e5b94b73d2"
Jul 17 04:50:28.878339 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:28.878299    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6bkt5"
Jul 17 04:50:28.878790 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:28.878345    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vmqd"
Jul 17 04:50:28.878790 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:28.878451    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6bkt5" podUID="237abccc-c19c-4fca-be94-d3013f3c2e86"
Jul 17 04:50:28.878790 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:28.878575    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vmqd" podUID="bcee811d-1474-45f6-b0c7-afceee510ccd"
Jul 17 04:50:29.878022 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:29.877990    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzdct"
Jul 17 04:50:29.878200 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:29.878118    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzdct" podUID="a54b09be-6889-45d6-a748-68e5b94b73d2"
Jul 17 04:50:30.438521 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:30.438486    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a54b09be-6889-45d6-a748-68e5b94b73d2-metrics-certs\") pod \"network-metrics-daemon-rzdct\" (UID: \"a54b09be-6889-45d6-a748-68e5b94b73d2\") " pod="openshift-multus/network-metrics-daemon-rzdct"
Jul 17 04:50:30.438935 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:30.438611    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 04:50:30.438935 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:30.438668    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a54b09be-6889-45d6-a748-68e5b94b73d2-metrics-certs podName:a54b09be-6889-45d6-a748-68e5b94b73d2 nodeName:}" failed. No retries permitted until 2026-07-17 04:50:46.43865361 +0000 UTC m=+34.122247245 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a54b09be-6889-45d6-a748-68e5b94b73d2-metrics-certs") pod "network-metrics-daemon-rzdct" (UID: "a54b09be-6889-45d6-a748-68e5b94b73d2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 04:50:30.539580 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:30.539498    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/237abccc-c19c-4fca-be94-d3013f3c2e86-original-pull-secret\") pod \"global-pull-secret-syncer-6bkt5\" (UID: \"237abccc-c19c-4fca-be94-d3013f3c2e86\") " pod="kube-system/global-pull-secret-syncer-6bkt5"
Jul 17 04:50:30.539742 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:30.539590    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4xwm\" (UniqueName: \"kubernetes.io/projected/bcee811d-1474-45f6-b0c7-afceee510ccd-kube-api-access-l4xwm\") pod \"network-check-target-9vmqd\" (UID: \"bcee811d-1474-45f6-b0c7-afceee510ccd\") " pod="openshift-network-diagnostics/network-check-target-9vmqd"
Jul 17 04:50:30.539742 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:30.539661    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 04:50:30.539742 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:30.539687    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 04:50:30.539742 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:30.539699    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 04:50:30.539742 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:30.539711    2562 projected.go:196] Error preparing data for projected volume kube-api-access-l4xwm for pod openshift-network-diagnostics/network-check-target-9vmqd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 04:50:30.539742 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:30.539725    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/237abccc-c19c-4fca-be94-d3013f3c2e86-original-pull-secret podName:237abccc-c19c-4fca-be94-d3013f3c2e86 nodeName:}" failed. No retries permitted until 2026-07-17 04:50:46.539707208 +0000 UTC m=+34.223300846 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/237abccc-c19c-4fca-be94-d3013f3c2e86-original-pull-secret") pod "global-pull-secret-syncer-6bkt5" (UID: "237abccc-c19c-4fca-be94-d3013f3c2e86") : object "kube-system"/"original-pull-secret" not registered
Jul 17 04:50:30.540068 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:30.539745    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bcee811d-1474-45f6-b0c7-afceee510ccd-kube-api-access-l4xwm podName:bcee811d-1474-45f6-b0c7-afceee510ccd nodeName:}" failed. No retries permitted until 2026-07-17 04:50:46.539733935 +0000 UTC m=+34.223327570 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-l4xwm" (UniqueName: "kubernetes.io/projected/bcee811d-1474-45f6-b0c7-afceee510ccd-kube-api-access-l4xwm") pod "network-check-target-9vmqd" (UID: "bcee811d-1474-45f6-b0c7-afceee510ccd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 04:50:30.878325 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:30.878295    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6bkt5"
Jul 17 04:50:30.878523 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:30.878421    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6bkt5" podUID="237abccc-c19c-4fca-be94-d3013f3c2e86"
Jul 17 04:50:30.878523 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:30.878494    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vmqd"
Jul 17 04:50:30.878628 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:30.878602    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vmqd" podUID="bcee811d-1474-45f6-b0c7-afceee510ccd"
Jul 17 04:50:31.878307 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:31.878270    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzdct"
Jul 17 04:50:31.878689 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:31.878377    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzdct" podUID="a54b09be-6889-45d6-a748-68e5b94b73d2"
Jul 17 04:50:32.878888 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:32.878847    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vmqd"
Jul 17 04:50:32.879360 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:32.878976    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vmqd" podUID="bcee811d-1474-45f6-b0c7-afceee510ccd"
Jul 17 04:50:32.879360 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:32.879025    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6bkt5"
Jul 17 04:50:32.879360 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:32.879120    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6bkt5" podUID="237abccc-c19c-4fca-be94-d3013f3c2e86"
Jul 17 04:50:33.878140 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:33.878105    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzdct"
Jul 17 04:50:33.878329 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:33.878248    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzdct" podUID="a54b09be-6889-45d6-a748-68e5b94b73d2"
Jul 17 04:50:34.878945 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:34.878641    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6bkt5"
Jul 17 04:50:34.879428 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:34.878989    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6bkt5" podUID="237abccc-c19c-4fca-be94-d3013f3c2e86"
Jul 17 04:50:34.879428 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:34.878664    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vmqd"
Jul 17 04:50:34.879428 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:34.879096    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vmqd" podUID="bcee811d-1474-45f6-b0c7-afceee510ccd"
Jul 17 04:50:35.037108 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:35.037074    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nbdgc" event={"ID":"da4d7ee5-31c8-465a-b5bd-5127ac635d08","Type":"ContainerStarted","Data":"7d9118618bf02008d048b474d0264cf44a853422f5f93c3af41ae0a2995aa10a"}
Jul 17 04:50:35.038640 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:35.038607    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lbjqq" event={"ID":"f4c22119-9902-4e1f-bc76-97d51253b16c","Type":"ContainerStarted","Data":"28f482f94c9e120289539e545c845396105aa503ef17474d595289d675b2b2bb"}
Jul 17 04:50:35.039845 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:35.039811    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4l4dn" event={"ID":"052c1550-58cb-4558-8b40-d9bd51a597b9","Type":"ContainerStarted","Data":"da3c9ff73dc15cfebdfcecfdd9a554a4b66fd8c975c79ef21440e191f959d8cd"}
Jul 17 04:50:35.041242 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:35.041218    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tx9gj_c1baeee2-350c-43c5-9a04-fb34378edb5c/ovn-acl-logging/0.log"
Jul 17 04:50:35.041604 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:35.041581    2562 generic.go:356] "Generic (PLEG): container finished" podID="c1baeee2-350c-43c5-9a04-fb34378edb5c" containerID="12d3b1894a7e31b0f8da48c9b5ddc2936df8d30dc22b3f51400416b9d7ff8860" exitCode=1
Jul 17 04:50:35.041706 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:35.041641    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj" event={"ID":"c1baeee2-350c-43c5-9a04-fb34378edb5c","Type":"ContainerDied","Data":"12d3b1894a7e31b0f8da48c9b5ddc2936df8d30dc22b3f51400416b9d7ff8860"}
Jul 17 04:50:35.041762 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:35.041720    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj" event={"ID":"c1baeee2-350c-43c5-9a04-fb34378edb5c","Type":"ContainerStarted","Data":"a44f7dc437ada6d227f1017d8048e328657f89685edc2606203b4ba23ccc5319"}
Jul 17 04:50:35.043332 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:35.043307    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bwdpn" event={"ID":"43b05be8-ccbf-412d-ad62-74596e1710fa","Type":"ContainerStarted","Data":"a0d2cf4b6597a1167ddca054ece0fcb9e1d03336b82a8f8d2bc31a3b2658870b"}
Jul 17 04:50:35.044846 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:35.044822    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-chbnw" event={"ID":"36349201-affc-4d02-9527-4f43167cdb1e","Type":"ContainerStarted","Data":"02215e4b25418b04c52cd307626c30cb4bfefe7fa7202eeff0d6c8d6628a53ac"}
Jul 17 04:50:35.046446 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:35.046420    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7tkkm" event={"ID":"7960477c-6164-45a9-a338-3063feb4a4a2","Type":"ContainerStarted","Data":"54179a0798636185d1fe5879def2c044248089a23b4776f01e66324abc94aa90"}
Jul 17 04:50:35.047952 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:35.047932    2562 generic.go:356] "Generic (PLEG): container finished" podID="01097a42-523b-4764-b6a6-bf531915f32f" containerID="14e47d0bb7bb12cce1e9dc6af1f1cf380e8ad2f10a507f45f27db32044716981" exitCode=0
Jul 17 04:50:35.048253 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:35.047978    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-644hn" event={"ID":"01097a42-523b-4764-b6a6-bf531915f32f","Type":"ContainerDied","Data":"14e47d0bb7bb12cce1e9dc6af1f1cf380e8ad2f10a507f45f27db32044716981"}
Jul 17 04:50:35.054900 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:35.054863    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-19.ec2.internal" podStartSLOduration=21.054851392 podStartE2EDuration="21.054851392s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 04:50:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 04:50:18.026459393 +0000 UTC m=+5.710053052" watchObservedRunningTime="2026-07-17 04:50:35.054851392 +0000 UTC m=+22.738445049"
Jul 17 04:50:35.055145 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:35.055112    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-nbdgc" podStartSLOduration=8.820394216 podStartE2EDuration="23.05508734s" totalImagesPullingTime="14.234693124s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 04:50:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 04:50:35.054807781 +0000 UTC m=+22.738401440" watchObservedRunningTime="2026-07-17 04:50:35.05508734 +0000 UTC m=+22.738680997"
Jul 17 04:50:35.090170 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:35.090096    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-chbnw" podStartSLOduration=4.414275633 podStartE2EDuration="23.090075799s" totalImagesPullingTime="18.675800166s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 04:50:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 04:50:35.090044713 +0000 UTC m=+22.773638389" watchObservedRunningTime="2026-07-17 04:50:35.090075799 +0000 UTC m=+22.773669458"
Jul 17 04:50:35.090483 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:35.090459    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-lbjqq" podStartSLOduration=4.374620022 podStartE2EDuration="23.090450129s" totalImagesPullingTime="18.715830107s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 04:50:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 04:50:35.072859387 +0000 UTC m=+22.756453045" watchObservedRunningTime="2026-07-17 04:50:35.090450129 +0000 UTC m=+22.774043786"
Jul 17 04:50:35.105361 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:35.105313    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-bwdpn" podStartSLOduration=4.434933619 podStartE2EDuration="23.10529916s" totalImagesPullingTime="18.670365541s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 04:50:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 04:50:35.104856536 +0000 UTC m=+22.788450193" watchObservedRunningTime="2026-07-17 04:50:35.10529916 +0000 UTC m=+22.788892816"
Jul 17 04:50:35.120540 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:35.120501    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-4l4dn" podStartSLOduration=3.463406275 podStartE2EDuration="22.120487105s" totalImagesPullingTime="18.65708083s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 04:50:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 04:50:35.120369686 +0000 UTC m=+22.803963344" watchObservedRunningTime="2026-07-17 04:50:35.120487105 +0000 UTC m=+22.804080762"
Jul 17 04:50:35.878480 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:35.878275    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzdct"
Jul 17 04:50:35.878669 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:35.878579    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzdct" podUID="a54b09be-6889-45d6-a748-68e5b94b73d2"
Jul 17 04:50:36.054129 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:36.054100    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tx9gj_c1baeee2-350c-43c5-9a04-fb34378edb5c/ovn-acl-logging/0.log"
Jul 17 04:50:36.055129 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:36.055063    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj" event={"ID":"c1baeee2-350c-43c5-9a04-fb34378edb5c","Type":"ContainerStarted","Data":"45255c9e8649a708c709b91b44eef6225186679c60059a2e9992ad3aad751d61"}
Jul 17 04:50:36.055205 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:36.055137    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj" event={"ID":"c1baeee2-350c-43c5-9a04-fb34378edb5c","Type":"ContainerStarted","Data":"e49a123b5ae8b8080ee0723dd78177587d7f52f4714fae7db511dbaa26b2fc08"}
Jul 17 04:50:36.055205 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:36.055187    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj" event={"ID":"c1baeee2-350c-43c5-9a04-fb34378edb5c","Type":"ContainerStarted","Data":"9b91369fef203c0e86eebb525a87e998837e5499fe1d8f016571949b3b1bd257"}
Jul 17 04:50:36.055310 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:36.055235    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj" event={"ID":"c1baeee2-350c-43c5-9a04-fb34378edb5c","Type":"ContainerStarted","Data":"a6758a10fe2848a0db85520fa8478e503e1fc9977a4cde89490fe2f96f6c5a87"}
Jul 17 04:50:36.437476 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:36.437440    2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 17 04:50:36.770850 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:36.770729    2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-17T04:50:36.437462297Z","UUID":"01051fb3-bb88-4b04-833b-9049dc8a7dcc","Handler":null,"Name":"","Endpoint":""}
Jul 17 04:50:36.772203 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:36.772182    2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 17 04:50:36.772331 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:36.772225    2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 17 04:50:36.877741 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:36.877713    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vmqd"
Jul 17 04:50:36.877908 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:36.877713    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6bkt5"
Jul 17 04:50:36.877908 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:36.877848    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vmqd" podUID="bcee811d-1474-45f6-b0c7-afceee510ccd"
Jul 17 04:50:36.878040 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:36.877930    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6bkt5" podUID="237abccc-c19c-4fca-be94-d3013f3c2e86"
Jul 17 04:50:37.059822 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:37.059731    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7tkkm" event={"ID":"7960477c-6164-45a9-a338-3063feb4a4a2","Type":"ContainerStarted","Data":"80d3daa37693a13cabf9827bbb57caae7973652be986b03a0e8fe6a181ee8bfb"}
Jul 17 04:50:37.061796 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:37.061758    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kfkjz" event={"ID":"14db56e5-42df-4de1-bb13-86d893eed9c0","Type":"ContainerStarted","Data":"30809f3e1bf4dc5ba5315a2ecabe3a807bbaabfed5618c29575860faf7381f56"}
Jul 17 04:50:37.878483 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:37.878459    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzdct"
Jul 17 04:50:37.878636 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:37.878556    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzdct" podUID="a54b09be-6889-45d6-a748-68e5b94b73d2"
Jul 17 04:50:38.065911 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:38.065869    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7tkkm" event={"ID":"7960477c-6164-45a9-a338-3063feb4a4a2","Type":"ContainerStarted","Data":"8514ad0985edb5215b9a838d647292e22942773a50b899294da0f726cf91fd6f"}
Jul 17 04:50:38.069549 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:38.069528    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tx9gj_c1baeee2-350c-43c5-9a04-fb34378edb5c/ovn-acl-logging/0.log"
Jul 17 04:50:38.069982 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:38.069940    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj" event={"ID":"c1baeee2-350c-43c5-9a04-fb34378edb5c","Type":"ContainerStarted","Data":"4c746e83e4052086f26e68365b62705769adc8546e932f0a77b4c2b542aa9d6c"}
Jul 17 04:50:38.092157 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:38.092109    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-kfkjz" podStartSLOduration=7.6154551470000005 podStartE2EDuration="26.092090675s" totalImagesPullingTime="18.476635528s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 04:50:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 04:50:37.079133856 +0000 UTC m=+24.762727514" watchObservedRunningTime="2026-07-17 04:50:38.092090675 +0000 UTC m=+25.775684389"
Jul 17 04:50:38.092383 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:38.092355    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7tkkm" podStartSLOduration=4.214983021 podStartE2EDuration="25.092350046s" totalImagesPullingTime="20.877367025s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 04:50:13 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 04:50:38.092044652 +0000 UTC m=+25.775638327" watchObservedRunningTime="2026-07-17 04:50:38.092350046 +0000 UTC m=+25.775943703"
Jul 17 04:50:38.878848 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:38.878660    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vmqd"
Jul 17 04:50:38.879040 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:38.878730    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6bkt5"
Jul 17 04:50:38.879040 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:38.878976    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vmqd" podUID="bcee811d-1474-45f6-b0c7-afceee510ccd"
Jul 17 04:50:38.879040 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:38.879028    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6bkt5" podUID="237abccc-c19c-4fca-be94-d3013f3c2e86"
Jul 17 04:50:39.448947 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:39.448909    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-nbdgc"
Jul 17 04:50:39.449724 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:39.449703    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-nbdgc"
Jul 17 04:50:39.878042 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:39.878013    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzdct"
Jul 17 04:50:39.878228 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:39.878117    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzdct" podUID="a54b09be-6889-45d6-a748-68e5b94b73d2"
Jul 17 04:50:40.078838 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:40.078813    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tx9gj_c1baeee2-350c-43c5-9a04-fb34378edb5c/ovn-acl-logging/0.log"
Jul 17 04:50:40.080872 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:40.080468    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj" event={"ID":"c1baeee2-350c-43c5-9a04-fb34378edb5c","Type":"ContainerStarted","Data":"7c462536d32568c2b9a9b3abfebcce4339c59fe339c1aff3ddbfe6a0398281d7"}
Jul 17 04:50:40.080872 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:40.080736    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:40.080872 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:40.080795    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:40.081400 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:40.080926    2562 scope.go:117] "RemoveContainer" containerID="12d3b1894a7e31b0f8da48c9b5ddc2936df8d30dc22b3f51400416b9d7ff8860"
Jul 17 04:50:40.084463 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:40.084439    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-644hn" event={"ID":"01097a42-523b-4764-b6a6-bf531915f32f","Type":"ContainerStarted","Data":"c00d5fe8e7ec76bb1969ee014bab5bf5798e6a4ee7bdb8b546c40b62d0b943cb"}
Jul 17 04:50:40.098939 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:40.098858    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:40.101121 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:40.100986    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:40.878310 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:40.878111    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vmqd"
Jul 17 04:50:40.879129 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:40.878111    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6bkt5"
Jul 17 04:50:40.879129 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:40.878386    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vmqd" podUID="bcee811d-1474-45f6-b0c7-afceee510ccd"
Jul 17 04:50:40.879129 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:40.878479    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6bkt5" podUID="237abccc-c19c-4fca-be94-d3013f3c2e86"
Jul 17 04:50:41.087696 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:41.087661    2562 generic.go:356] "Generic (PLEG): container finished" podID="01097a42-523b-4764-b6a6-bf531915f32f" containerID="c00d5fe8e7ec76bb1969ee014bab5bf5798e6a4ee7bdb8b546c40b62d0b943cb" exitCode=0
Jul 17 04:50:41.087843 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:41.087756    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-644hn" event={"ID":"01097a42-523b-4764-b6a6-bf531915f32f","Type":"ContainerDied","Data":"c00d5fe8e7ec76bb1969ee014bab5bf5798e6a4ee7bdb8b546c40b62d0b943cb"}
Jul 17 04:50:41.091459 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:41.091441    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tx9gj_c1baeee2-350c-43c5-9a04-fb34378edb5c/ovn-acl-logging/0.log"
Jul 17 04:50:41.091776 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:41.091756    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj" event={"ID":"c1baeee2-350c-43c5-9a04-fb34378edb5c","Type":"ContainerStarted","Data":"64ecddf2d3967afbee99efed3ba8849abbc2881ca24b123c01b7186784d40c2f"}
Jul 17 04:50:41.091892 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:41.091880    2562 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 17 04:50:41.136876 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:41.136837    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj" podStartSLOduration=9.378420527 podStartE2EDuration="28.136823597s" totalImagesPullingTime="18.75840307s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 04:50:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 04:50:41.136330498 +0000 UTC m=+28.819924155" watchObservedRunningTime="2026-07-17 04:50:41.136823597 +0000 UTC m=+28.820417253"
Jul 17 04:50:41.280378 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:41.280346    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:50:41.878615 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:41.878580    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzdct"
Jul 17 04:50:41.878991 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:41.878680    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzdct" podUID="a54b09be-6889-45d6-a748-68e5b94b73d2"
Jul 17 04:50:41.943740 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:41.943717    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-nbdgc"
Jul 17 04:50:41.943857 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:41.943845    2562 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 17 04:50:41.944305 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:41.944290    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-nbdgc"
Jul 17 04:50:42.095622 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:42.095537    2562 generic.go:356] "Generic (PLEG): container finished" podID="01097a42-523b-4764-b6a6-bf531915f32f" containerID="e45030892c476bc47bb55c23190febaca15124236918c36b65cf59e16fec3bc8" exitCode=0
Jul 17 04:50:42.095622 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:42.095606    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-644hn" event={"ID":"01097a42-523b-4764-b6a6-bf531915f32f","Type":"ContainerDied","Data":"e45030892c476bc47bb55c23190febaca15124236918c36b65cf59e16fec3bc8"}
Jul 17 04:50:42.878640 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:42.878613    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6bkt5"
Jul 17 04:50:42.878938 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:42.878713    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6bkt5" podUID="237abccc-c19c-4fca-be94-d3013f3c2e86"
Jul 17 04:50:42.878938 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:42.878764    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vmqd"
Jul 17 04:50:42.878938 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:42.878841    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vmqd" podUID="bcee811d-1474-45f6-b0c7-afceee510ccd"
Jul 17 04:50:43.100426 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:43.100398    2562 generic.go:356] "Generic (PLEG): container finished" podID="01097a42-523b-4764-b6a6-bf531915f32f" containerID="09bb40b44b6bbe44bf57eca50dfef20000c0bc62c3c3e992f1da2dd5b5499702" exitCode=0
Jul 17 04:50:43.100578 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:43.100483    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-644hn" event={"ID":"01097a42-523b-4764-b6a6-bf531915f32f","Type":"ContainerDied","Data":"09bb40b44b6bbe44bf57eca50dfef20000c0bc62c3c3e992f1da2dd5b5499702"}
Jul 17 04:50:43.878028 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:43.877995    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzdct"
Jul 17 04:50:43.878190 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:43.878113    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzdct" podUID="a54b09be-6889-45d6-a748-68e5b94b73d2"
Jul 17 04:50:44.878708 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:44.878673    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6bkt5"
Jul 17 04:50:44.879213 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:44.878720    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vmqd"
Jul 17 04:50:44.879213 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:44.878822    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6bkt5" podUID="237abccc-c19c-4fca-be94-d3013f3c2e86"
Jul 17 04:50:44.879340 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:44.879250    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vmqd" podUID="bcee811d-1474-45f6-b0c7-afceee510ccd"
Jul 17 04:50:45.878401 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:45.878369    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzdct"
Jul 17 04:50:45.878595 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:45.878499    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzdct" podUID="a54b09be-6889-45d6-a748-68e5b94b73d2"
Jul 17 04:50:46.459692 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:46.459653    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a54b09be-6889-45d6-a748-68e5b94b73d2-metrics-certs\") pod \"network-metrics-daemon-rzdct\" (UID: \"a54b09be-6889-45d6-a748-68e5b94b73d2\") " pod="openshift-multus/network-metrics-daemon-rzdct"
Jul 17 04:50:46.460307 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:46.459818    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 04:50:46.460307 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:46.459905    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a54b09be-6889-45d6-a748-68e5b94b73d2-metrics-certs podName:a54b09be-6889-45d6-a748-68e5b94b73d2 nodeName:}" failed. No retries permitted until 2026-07-17 04:51:18.45988058 +0000 UTC m=+66.143474222 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a54b09be-6889-45d6-a748-68e5b94b73d2-metrics-certs") pod "network-metrics-daemon-rzdct" (UID: "a54b09be-6889-45d6-a748-68e5b94b73d2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 04:50:46.560569 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:46.560536    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4xwm\" (UniqueName: \"kubernetes.io/projected/bcee811d-1474-45f6-b0c7-afceee510ccd-kube-api-access-l4xwm\") pod \"network-check-target-9vmqd\" (UID: \"bcee811d-1474-45f6-b0c7-afceee510ccd\") " pod="openshift-network-diagnostics/network-check-target-9vmqd"
Jul 17 04:50:46.560569 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:46.560577    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/237abccc-c19c-4fca-be94-d3013f3c2e86-original-pull-secret\") pod \"global-pull-secret-syncer-6bkt5\" (UID: \"237abccc-c19c-4fca-be94-d3013f3c2e86\") " pod="kube-system/global-pull-secret-syncer-6bkt5"
Jul 17 04:50:46.560806 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:46.560692    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 04:50:46.560806 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:46.560715    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 04:50:46.560806 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:46.560739    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 04:50:46.560806 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:46.560753    2562 projected.go:196] Error preparing data for projected volume kube-api-access-l4xwm for pod openshift-network-diagnostics/network-check-target-9vmqd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 04:50:46.560806 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:46.560759    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/237abccc-c19c-4fca-be94-d3013f3c2e86-original-pull-secret podName:237abccc-c19c-4fca-be94-d3013f3c2e86 nodeName:}" failed. No retries permitted until 2026-07-17 04:51:18.560741131 +0000 UTC m=+66.244334766 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/237abccc-c19c-4fca-be94-d3013f3c2e86-original-pull-secret") pod "global-pull-secret-syncer-6bkt5" (UID: "237abccc-c19c-4fca-be94-d3013f3c2e86") : object "kube-system"/"original-pull-secret" not registered
Jul 17 04:50:46.560806 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:46.560807    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bcee811d-1474-45f6-b0c7-afceee510ccd-kube-api-access-l4xwm podName:bcee811d-1474-45f6-b0c7-afceee510ccd nodeName:}" failed. No retries permitted until 2026-07-17 04:51:18.560789183 +0000 UTC m=+66.244382831 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-l4xwm" (UniqueName: "kubernetes.io/projected/bcee811d-1474-45f6-b0c7-afceee510ccd-kube-api-access-l4xwm") pod "network-check-target-9vmqd" (UID: "bcee811d-1474-45f6-b0c7-afceee510ccd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 04:50:46.878546 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:46.878494    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6bkt5"
Jul 17 04:50:46.878737 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:46.878629    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6bkt5" podUID="237abccc-c19c-4fca-be94-d3013f3c2e86"
Jul 17 04:50:46.878737 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:46.878685    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vmqd"
Jul 17 04:50:46.878850 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:46.878800    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vmqd" podUID="bcee811d-1474-45f6-b0c7-afceee510ccd"
Jul 17 04:50:47.878463 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:47.878441    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzdct"
Jul 17 04:50:47.878769 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:47.878553    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzdct" podUID="a54b09be-6889-45d6-a748-68e5b94b73d2"
Jul 17 04:50:48.115061 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:48.115030    2562 generic.go:356] "Generic (PLEG): container finished" podID="01097a42-523b-4764-b6a6-bf531915f32f" containerID="f030884a26d2e977975d323c4b2726a27497337ac00ad305b27150739b7ea2cb" exitCode=0
Jul 17 04:50:48.115222 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:48.115067    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-644hn" event={"ID":"01097a42-523b-4764-b6a6-bf531915f32f","Type":"ContainerDied","Data":"f030884a26d2e977975d323c4b2726a27497337ac00ad305b27150739b7ea2cb"}
Jul 17 04:50:48.878044 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:48.877826    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6bkt5"
Jul 17 04:50:48.878221 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:48.877891    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vmqd"
Jul 17 04:50:48.878221 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:48.878136    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6bkt5" podUID="237abccc-c19c-4fca-be94-d3013f3c2e86"
Jul 17 04:50:48.878221 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:48.878190    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vmqd" podUID="bcee811d-1474-45f6-b0c7-afceee510ccd"
Jul 17 04:50:49.120156 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:49.120119    2562 generic.go:356] "Generic (PLEG): container finished" podID="01097a42-523b-4764-b6a6-bf531915f32f" containerID="e90dd5a5d285d1bcbcd0de67292bd422004633a519844ead5a007c9634d11344" exitCode=0
Jul 17 04:50:49.120494 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:49.120188    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-644hn" event={"ID":"01097a42-523b-4764-b6a6-bf531915f32f","Type":"ContainerDied","Data":"e90dd5a5d285d1bcbcd0de67292bd422004633a519844ead5a007c9634d11344"}
Jul 17 04:50:49.877677 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:49.877645    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzdct"
Jul 17 04:50:49.877841 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:49.877746    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzdct" podUID="a54b09be-6889-45d6-a748-68e5b94b73d2"
Jul 17 04:50:50.126196 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:50.126162    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-644hn" event={"ID":"01097a42-523b-4764-b6a6-bf531915f32f","Type":"ContainerStarted","Data":"3389bf7a7b67525b55577abacd9442c530152e922afca45db642de7a4cc0682f"}
Jul 17 04:50:50.152264 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:50.152157    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-644hn" podStartSLOduration=8.541930333 podStartE2EDuration="38.152141332s" totalImagesPullingTime="29.610210999s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 04:50:12 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 04:50:50.150282892 +0000 UTC m=+37.833876548" watchObservedRunningTime="2026-07-17 04:50:50.152141332 +0000 UTC m=+37.835734989"
Jul 17 04:50:50.877627 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:50.877595    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6bkt5"
Jul 17 04:50:50.877809 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:50.877605    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vmqd"
Jul 17 04:50:50.877809 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:50.877695    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6bkt5" podUID="237abccc-c19c-4fca-be94-d3013f3c2e86"
Jul 17 04:50:50.877809 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:50.877755    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vmqd" podUID="bcee811d-1474-45f6-b0c7-afceee510ccd"
Jul 17 04:50:51.878268 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:51.878231    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzdct"
Jul 17 04:50:51.878703 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:51.878365    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzdct" podUID="a54b09be-6889-45d6-a748-68e5b94b73d2"
Jul 17 04:50:52.879304 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:52.879268    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vmqd"
Jul 17 04:50:52.879687 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:52.879342    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6bkt5"
Jul 17 04:50:52.879687 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:52.879357    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vmqd" podUID="bcee811d-1474-45f6-b0c7-afceee510ccd"
Jul 17 04:50:52.879687 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:52.879432    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6bkt5" podUID="237abccc-c19c-4fca-be94-d3013f3c2e86"
Jul 17 04:50:53.878214 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:53.878179    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzdct"
Jul 17 04:50:53.878380 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:53.878294    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzdct" podUID="a54b09be-6889-45d6-a748-68e5b94b73d2"
Jul 17 04:50:54.878251 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:54.878216    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vmqd"
Jul 17 04:50:54.878615 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:54.878319    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vmqd" podUID="bcee811d-1474-45f6-b0c7-afceee510ccd"
Jul 17 04:50:54.878615 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:54.878375    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6bkt5"
Jul 17 04:50:54.878615 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:54.878453    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6bkt5" podUID="237abccc-c19c-4fca-be94-d3013f3c2e86"
Jul 17 04:50:55.877884 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:55.877853    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzdct"
Jul 17 04:50:55.878091 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:55.877952    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzdct" podUID="a54b09be-6889-45d6-a748-68e5b94b73d2"
Jul 17 04:50:56.877699 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:56.877664    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6bkt5"
Jul 17 04:50:56.878184 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:56.877704    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vmqd"
Jul 17 04:50:56.878184 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:56.877779    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6bkt5" podUID="237abccc-c19c-4fca-be94-d3013f3c2e86"
Jul 17 04:50:56.878184 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:56.877880    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vmqd" podUID="bcee811d-1474-45f6-b0c7-afceee510ccd"
Jul 17 04:50:57.877850 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:57.877824    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzdct"
Jul 17 04:50:57.878443 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:57.877928    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzdct" podUID="a54b09be-6889-45d6-a748-68e5b94b73d2"
Jul 17 04:50:58.017451 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:58.017417    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9vmqd"]
Jul 17 04:50:58.017630 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:58.017526    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vmqd"
Jul 17 04:50:58.017630 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:58.017609    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vmqd" podUID="bcee811d-1474-45f6-b0c7-afceee510ccd"
Jul 17 04:50:58.020636 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:58.020608    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6bkt5"]
Jul 17 04:50:58.020775 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:58.020713    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6bkt5"
Jul 17 04:50:58.020851 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:58.020819    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6bkt5" podUID="237abccc-c19c-4fca-be94-d3013f3c2e86"
Jul 17 04:50:58.021382 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:58.021344    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rzdct"]
Jul 17 04:50:58.143154 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:58.143119    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzdct"
Jul 17 04:50:58.143331 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:58.143247    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzdct" podUID="a54b09be-6889-45d6-a748-68e5b94b73d2"
Jul 17 04:50:59.877643 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:59.877597    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6bkt5"
Jul 17 04:50:59.877643 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:59.877630    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzdct"
Jul 17 04:50:59.877643 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:50:59.877647    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vmqd"
Jul 17 04:50:59.878143 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:59.877739    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6bkt5" podUID="237abccc-c19c-4fca-be94-d3013f3c2e86"
Jul 17 04:50:59.878143 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:59.877865    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzdct" podUID="a54b09be-6889-45d6-a748-68e5b94b73d2"
Jul 17 04:50:59.878143 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:50:59.877911    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vmqd" podUID="bcee811d-1474-45f6-b0c7-afceee510ccd"
Jul 17 04:51:01.877935 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:01.877750    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6bkt5"
Jul 17 04:51:01.878423 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:01.877770    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzdct"
Jul 17 04:51:01.878423 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:51:01.878040    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6bkt5" podUID="237abccc-c19c-4fca-be94-d3013f3c2e86"
Jul 17 04:51:01.878423 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:01.877808    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vmqd"
Jul 17 04:51:01.878423 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:51:01.878096    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzdct" podUID="a54b09be-6889-45d6-a748-68e5b94b73d2"
Jul 17 04:51:01.878423 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:51:01.878150    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9vmqd" podUID="bcee811d-1474-45f6-b0c7-afceee510ccd"
Jul 17 04:51:02.646874 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.646847    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-19.ec2.internal" event="NodeReady"
Jul 17 04:51:02.647132 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.647004    2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 17 04:51:02.694018 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.693981    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-kf2nj"]
Jul 17 04:51:02.714572 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.714538    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-8pbjc"]
Jul 17 04:51:02.714717 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.714666    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kf2nj"
Jul 17 04:51:02.717416 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.717392    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 17 04:51:02.717416 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.717425    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 17 04:51:02.717608 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.717478    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kswzc\""
Jul 17 04:51:02.734949 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.734918    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-kf2nj"]
Jul 17 04:51:02.735101 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.734983    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8pbjc"]
Jul 17 04:51:02.735101 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.735051    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8pbjc"
Jul 17 04:51:02.737843 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.737818    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 17 04:51:02.738088 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.737885    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-swxjd\""
Jul 17 04:51:02.738088 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.737898    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 17 04:51:02.738088 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.737885    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 17 04:51:02.801249 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.801220    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-pwfdn"]
Jul 17 04:51:02.820675 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.820648    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pwfdn"]
Jul 17 04:51:02.820812 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.820756    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pwfdn"
Jul 17 04:51:02.823665 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.823614    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 17 04:51:02.823808 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.823677    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 17 04:51:02.823808 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.823721    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kdz6x\""
Jul 17 04:51:02.823808 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.823744    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 17 04:51:02.823956 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.823861    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 17 04:51:02.881015 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.880951    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbthv\" (UniqueName: \"kubernetes.io/projected/3f3f79dc-2048-413f-a701-1ac6a22554a4-kube-api-access-lbthv\") pod \"ingress-canary-8pbjc\" (UID: \"3f3f79dc-2048-413f-a701-1ac6a22554a4\") " pod="openshift-ingress-canary/ingress-canary-8pbjc"
Jul 17 04:51:02.881470 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.881043    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b4a2877-135f-4be6-a249-83c2006db349-metrics-tls\") pod \"dns-default-kf2nj\" (UID: \"9b4a2877-135f-4be6-a249-83c2006db349\") " pod="openshift-dns/dns-default-kf2nj"
Jul 17 04:51:02.881470 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.881072    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b4a2877-135f-4be6-a249-83c2006db349-tmp-dir\") pod \"dns-default-kf2nj\" (UID: \"9b4a2877-135f-4be6-a249-83c2006db349\") " pod="openshift-dns/dns-default-kf2nj"
Jul 17 04:51:02.881470 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.881105    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpc75\" (UniqueName: \"kubernetes.io/projected/9b4a2877-135f-4be6-a249-83c2006db349-kube-api-access-tpc75\") pod \"dns-default-kf2nj\" (UID: \"9b4a2877-135f-4be6-a249-83c2006db349\") " pod="openshift-dns/dns-default-kf2nj"
Jul 17 04:51:02.881470 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.881179    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3f3f79dc-2048-413f-a701-1ac6a22554a4-cert\") pod \"ingress-canary-8pbjc\" (UID: \"3f3f79dc-2048-413f-a701-1ac6a22554a4\") " pod="openshift-ingress-canary/ingress-canary-8pbjc"
Jul 17 04:51:02.881470 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.881209    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9b4a2877-135f-4be6-a249-83c2006db349-config-volume\") pod \"dns-default-kf2nj\" (UID: \"9b4a2877-135f-4be6-a249-83c2006db349\") " pod="openshift-dns/dns-default-kf2nj"
Jul 17 04:51:02.982563 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.982462    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbthv\" (UniqueName: \"kubernetes.io/projected/3f3f79dc-2048-413f-a701-1ac6a22554a4-kube-api-access-lbthv\") pod \"ingress-canary-8pbjc\" (UID: \"3f3f79dc-2048-413f-a701-1ac6a22554a4\") " pod="openshift-ingress-canary/ingress-canary-8pbjc"
Jul 17 04:51:02.982563 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.982515    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b4a2877-135f-4be6-a249-83c2006db349-metrics-tls\") pod \"dns-default-kf2nj\" (UID: \"9b4a2877-135f-4be6-a249-83c2006db349\") " pod="openshift-dns/dns-default-kf2nj"
Jul 17 04:51:02.982563 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.982541    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b4a2877-135f-4be6-a249-83c2006db349-tmp-dir\") pod \"dns-default-kf2nj\" (UID: \"9b4a2877-135f-4be6-a249-83c2006db349\") " pod="openshift-dns/dns-default-kf2nj"
Jul 17 04:51:02.982835 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.982662    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpc75\" (UniqueName: \"kubernetes.io/projected/9b4a2877-135f-4be6-a249-83c2006db349-kube-api-access-tpc75\") pod \"dns-default-kf2nj\" (UID: \"9b4a2877-135f-4be6-a249-83c2006db349\") " pod="openshift-dns/dns-default-kf2nj"
Jul 17 04:51:02.982835 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.982697    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5ddbfecb-13e8-4c64-942b-6b8100d6b92a-data-volume\") pod \"insights-runtime-extractor-pwfdn\" (UID: \"5ddbfecb-13e8-4c64-942b-6b8100d6b92a\") " pod="openshift-insights/insights-runtime-extractor-pwfdn"
Jul 17 04:51:02.982835 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.982725    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jh7s6\" (UniqueName: \"kubernetes.io/projected/5ddbfecb-13e8-4c64-942b-6b8100d6b92a-kube-api-access-jh7s6\") pod \"insights-runtime-extractor-pwfdn\" (UID: \"5ddbfecb-13e8-4c64-942b-6b8100d6b92a\") " pod="openshift-insights/insights-runtime-extractor-pwfdn"
Jul 17 04:51:02.982835 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.982758    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5ddbfecb-13e8-4c64-942b-6b8100d6b92a-crio-socket\") pod \"insights-runtime-extractor-pwfdn\" (UID: \"5ddbfecb-13e8-4c64-942b-6b8100d6b92a\") " pod="openshift-insights/insights-runtime-extractor-pwfdn"
Jul 17 04:51:02.982835 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.982779    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5ddbfecb-13e8-4c64-942b-6b8100d6b92a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pwfdn\" (UID: \"5ddbfecb-13e8-4c64-942b-6b8100d6b92a\") " pod="openshift-insights/insights-runtime-extractor-pwfdn"
Jul 17 04:51:02.982835 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.982821    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3f3f79dc-2048-413f-a701-1ac6a22554a4-cert\") pod \"ingress-canary-8pbjc\" (UID: \"3f3f79dc-2048-413f-a701-1ac6a22554a4\") " pod="openshift-ingress-canary/ingress-canary-8pbjc"
Jul 17 04:51:02.983167 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.982848    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9b4a2877-135f-4be6-a249-83c2006db349-config-volume\") pod \"dns-default-kf2nj\" (UID: \"9b4a2877-135f-4be6-a249-83c2006db349\") " pod="openshift-dns/dns-default-kf2nj"
Jul 17 04:51:02.983167 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.982875    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b4a2877-135f-4be6-a249-83c2006db349-tmp-dir\") pod \"dns-default-kf2nj\" (UID: \"9b4a2877-135f-4be6-a249-83c2006db349\") " pod="openshift-dns/dns-default-kf2nj"
Jul 17 04:51:02.983167 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.982881    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5ddbfecb-13e8-4c64-942b-6b8100d6b92a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pwfdn\" (UID: \"5ddbfecb-13e8-4c64-942b-6b8100d6b92a\") " pod="openshift-insights/insights-runtime-extractor-pwfdn"
Jul 17 04:51:02.983456 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.983437    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9b4a2877-135f-4be6-a249-83c2006db349-config-volume\") pod \"dns-default-kf2nj\" (UID: \"9b4a2877-135f-4be6-a249-83c2006db349\") " pod="openshift-dns/dns-default-kf2nj"
Jul 17 04:51:02.986713 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.986694    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b4a2877-135f-4be6-a249-83c2006db349-metrics-tls\") pod \"dns-default-kf2nj\" (UID: \"9b4a2877-135f-4be6-a249-83c2006db349\") " pod="openshift-dns/dns-default-kf2nj"
Jul 17 04:51:02.986775 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.986757    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3f3f79dc-2048-413f-a701-1ac6a22554a4-cert\") pod \"ingress-canary-8pbjc\" (UID: \"3f3f79dc-2048-413f-a701-1ac6a22554a4\") " pod="openshift-ingress-canary/ingress-canary-8pbjc"
Jul 17 04:51:02.990254 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.990224    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpc75\" (UniqueName: \"kubernetes.io/projected/9b4a2877-135f-4be6-a249-83c2006db349-kube-api-access-tpc75\") pod \"dns-default-kf2nj\" (UID: \"9b4a2877-135f-4be6-a249-83c2006db349\") " pod="openshift-dns/dns-default-kf2nj"
Jul 17 04:51:02.990637 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:02.990615    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbthv\" (UniqueName: \"kubernetes.io/projected/3f3f79dc-2048-413f-a701-1ac6a22554a4-kube-api-access-lbthv\") pod \"ingress-canary-8pbjc\" (UID: \"3f3f79dc-2048-413f-a701-1ac6a22554a4\") " pod="openshift-ingress-canary/ingress-canary-8pbjc"
Jul 17 04:51:03.023426 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:03.023399    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kf2nj"
Jul 17 04:51:03.043511 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:03.043488    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8pbjc"
Jul 17 04:51:03.083633 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:03.083609    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5ddbfecb-13e8-4c64-942b-6b8100d6b92a-data-volume\") pod \"insights-runtime-extractor-pwfdn\" (UID: \"5ddbfecb-13e8-4c64-942b-6b8100d6b92a\") " pod="openshift-insights/insights-runtime-extractor-pwfdn"
Jul 17 04:51:03.083731 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:03.083645    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jh7s6\" (UniqueName: \"kubernetes.io/projected/5ddbfecb-13e8-4c64-942b-6b8100d6b92a-kube-api-access-jh7s6\") pod \"insights-runtime-extractor-pwfdn\" (UID: \"5ddbfecb-13e8-4c64-942b-6b8100d6b92a\") " pod="openshift-insights/insights-runtime-extractor-pwfdn"
Jul 17 04:51:03.083731 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:03.083670    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5ddbfecb-13e8-4c64-942b-6b8100d6b92a-crio-socket\") pod \"insights-runtime-extractor-pwfdn\" (UID: \"5ddbfecb-13e8-4c64-942b-6b8100d6b92a\") " pod="openshift-insights/insights-runtime-extractor-pwfdn"
Jul 17 04:51:03.083731 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:03.083699    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5ddbfecb-13e8-4c64-942b-6b8100d6b92a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pwfdn\" (UID: \"5ddbfecb-13e8-4c64-942b-6b8100d6b92a\") " pod="openshift-insights/insights-runtime-extractor-pwfdn"
Jul 17 04:51:03.083876 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:03.083738    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5ddbfecb-13e8-4c64-942b-6b8100d6b92a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pwfdn\" (UID: \"5ddbfecb-13e8-4c64-942b-6b8100d6b92a\") " pod="openshift-insights/insights-runtime-extractor-pwfdn"
Jul 17 04:51:03.083923 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:03.083901    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5ddbfecb-13e8-4c64-942b-6b8100d6b92a-crio-socket\") pod \"insights-runtime-extractor-pwfdn\" (UID: \"5ddbfecb-13e8-4c64-942b-6b8100d6b92a\") " pod="openshift-insights/insights-runtime-extractor-pwfdn"
Jul 17 04:51:03.084067 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:03.084037    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5ddbfecb-13e8-4c64-942b-6b8100d6b92a-data-volume\") pod \"insights-runtime-extractor-pwfdn\" (UID: \"5ddbfecb-13e8-4c64-942b-6b8100d6b92a\") " pod="openshift-insights/insights-runtime-extractor-pwfdn"
Jul 17 04:51:03.084306 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:03.084286    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5ddbfecb-13e8-4c64-942b-6b8100d6b92a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pwfdn\" (UID: \"5ddbfecb-13e8-4c64-942b-6b8100d6b92a\") " pod="openshift-insights/insights-runtime-extractor-pwfdn"
Jul 17 04:51:03.086654 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:03.086635    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5ddbfecb-13e8-4c64-942b-6b8100d6b92a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pwfdn\" (UID: \"5ddbfecb-13e8-4c64-942b-6b8100d6b92a\") " pod="openshift-insights/insights-runtime-extractor-pwfdn"
Jul 17 04:51:03.091972 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:03.091939    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jh7s6\" (UniqueName: \"kubernetes.io/projected/5ddbfecb-13e8-4c64-942b-6b8100d6b92a-kube-api-access-jh7s6\") pod \"insights-runtime-extractor-pwfdn\" (UID: \"5ddbfecb-13e8-4c64-942b-6b8100d6b92a\") " pod="openshift-insights/insights-runtime-extractor-pwfdn"
Jul 17 04:51:03.132212 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:03.131941    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pwfdn"
Jul 17 04:51:03.224799 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:03.224763    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-kf2nj"]
Jul 17 04:51:03.228498 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:03.228477    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8pbjc"]
Jul 17 04:51:03.230145 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:51:03.230120    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b4a2877_135f_4be6_a249_83c2006db349.slice/crio-63525df02518a2717068bfee677d3f41f1fb03b8e8bd521146c01637f8743e60 WatchSource:0}: Error finding container 63525df02518a2717068bfee677d3f41f1fb03b8e8bd521146c01637f8743e60: Status 404 returned error can't find the container with id 63525df02518a2717068bfee677d3f41f1fb03b8e8bd521146c01637f8743e60
Jul 17 04:51:03.231278 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:51:03.231249    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f3f79dc_2048_413f_a701_1ac6a22554a4.slice/crio-4fc094ee04c58cf5702e23e7bf5c256a0684f39b2c0fe78f8a11b22ff56474cf WatchSource:0}: Error finding container 4fc094ee04c58cf5702e23e7bf5c256a0684f39b2c0fe78f8a11b22ff56474cf: Status 404 returned error can't find the container with id 4fc094ee04c58cf5702e23e7bf5c256a0684f39b2c0fe78f8a11b22ff56474cf
Jul 17 04:51:03.293860 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:03.293832    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pwfdn"]
Jul 17 04:51:03.296755 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:51:03.296731    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ddbfecb_13e8_4c64_942b_6b8100d6b92a.slice/crio-93b150937c25fd925519ad5344aa46db2d523d570e3820b69f6796520b5c68ec WatchSource:0}: Error finding container 93b150937c25fd925519ad5344aa46db2d523d570e3820b69f6796520b5c68ec: Status 404 returned error can't find the container with id 93b150937c25fd925519ad5344aa46db2d523d570e3820b69f6796520b5c68ec
Jul 17 04:51:03.657462 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:03.657421    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-kxf2j"]
Jul 17 04:51:03.670311 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:03.670284    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-kxf2j"]
Jul 17 04:51:03.670473 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:03.670396    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-kxf2j"
Jul 17 04:51:03.673261 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:03.672943    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 17 04:51:03.673261 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:03.673205    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-859td\""
Jul 17 04:51:03.788888 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:03.788856    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/87aa6ee8-0708-428a-985b-40dbd41f57cb-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-kxf2j\" (UID: \"87aa6ee8-0708-428a-985b-40dbd41f57cb\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-kxf2j"
Jul 17 04:51:03.878175 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:03.878136    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6bkt5"
Jul 17 04:51:03.878477 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:03.878136    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vmqd"
Jul 17 04:51:03.878987 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:03.878946    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzdct"
Jul 17 04:51:03.881459 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:03.881342    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 17 04:51:03.881810 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:03.881740    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 17 04:51:03.881988 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:03.881954    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mhn7j\""
Jul 17 04:51:03.882252 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:03.882233    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 17 04:51:03.882452 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:03.882436    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 17 04:51:03.882777 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:03.882758    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zwbtr\""
Jul 17 04:51:03.889259 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:03.889235    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/87aa6ee8-0708-428a-985b-40dbd41f57cb-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-kxf2j\" (UID: \"87aa6ee8-0708-428a-985b-40dbd41f57cb\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-kxf2j"
Jul 17 04:51:03.889367 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:51:03.889346    2562 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 17 04:51:03.889432 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:51:03.889418    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87aa6ee8-0708-428a-985b-40dbd41f57cb-tls-certificates podName:87aa6ee8-0708-428a-985b-40dbd41f57cb nodeName:}" failed. No retries permitted until 2026-07-17 04:51:04.389398411 +0000 UTC m=+52.072992048 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/87aa6ee8-0708-428a-985b-40dbd41f57cb-tls-certificates") pod "prometheus-operator-admission-webhook-85574b5f49-kxf2j" (UID: "87aa6ee8-0708-428a-985b-40dbd41f57cb") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 17 04:51:04.160697 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:04.160643    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pwfdn" event={"ID":"5ddbfecb-13e8-4c64-942b-6b8100d6b92a","Type":"ContainerStarted","Data":"93b150937c25fd925519ad5344aa46db2d523d570e3820b69f6796520b5c68ec"}
Jul 17 04:51:04.162250 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:04.162019    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8pbjc" event={"ID":"3f3f79dc-2048-413f-a701-1ac6a22554a4","Type":"ContainerStarted","Data":"4fc094ee04c58cf5702e23e7bf5c256a0684f39b2c0fe78f8a11b22ff56474cf"}
Jul 17 04:51:04.163530 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:04.163506    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kf2nj" event={"ID":"9b4a2877-135f-4be6-a249-83c2006db349","Type":"ContainerStarted","Data":"63525df02518a2717068bfee677d3f41f1fb03b8e8bd521146c01637f8743e60"}
Jul 17 04:51:04.394553 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:04.394169    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/87aa6ee8-0708-428a-985b-40dbd41f57cb-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-kxf2j\" (UID: \"87aa6ee8-0708-428a-985b-40dbd41f57cb\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-kxf2j"
Jul 17 04:51:04.414927 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:04.414825    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/87aa6ee8-0708-428a-985b-40dbd41f57cb-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-kxf2j\" (UID: \"87aa6ee8-0708-428a-985b-40dbd41f57cb\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-kxf2j"
Jul 17 04:51:04.582003 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:04.581440    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-kxf2j"
Jul 17 04:51:04.777992 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:04.777898    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-75ff58c455-pksb8"]
Jul 17 04:51:04.817667 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:04.817605    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75ff58c455-pksb8"]
Jul 17 04:51:04.817886 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:04.817732    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75ff58c455-pksb8"
Jul 17 04:51:04.820716 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:04.820699    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 17 04:51:04.820986 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:04.820952    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 17 04:51:04.821376 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:04.821353    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 17 04:51:04.823070 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:04.822043    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 17 04:51:04.823070 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:04.822213    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 17 04:51:04.823070 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:04.822289    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 17 04:51:04.823070 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:04.822501    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-hq762\""
Jul 17 04:51:04.829210 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:04.824828    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 17 04:51:04.831391 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:04.831364    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 17 04:51:04.999286 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:04.999249    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a143a72-1103-45f5-97b1-18bcb51c06bd-console-config\") pod \"console-75ff58c455-pksb8\" (UID: \"5a143a72-1103-45f5-97b1-18bcb51c06bd\") " pod="openshift-console/console-75ff58c455-pksb8"
Jul 17 04:51:04.999286 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:04.999291    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njq58\" (UniqueName: \"kubernetes.io/projected/5a143a72-1103-45f5-97b1-18bcb51c06bd-kube-api-access-njq58\") pod \"console-75ff58c455-pksb8\" (UID: \"5a143a72-1103-45f5-97b1-18bcb51c06bd\") " pod="openshift-console/console-75ff58c455-pksb8"
Jul 17 04:51:04.999771 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:04.999317    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a143a72-1103-45f5-97b1-18bcb51c06bd-trusted-ca-bundle\") pod \"console-75ff58c455-pksb8\" (UID: \"5a143a72-1103-45f5-97b1-18bcb51c06bd\") " pod="openshift-console/console-75ff58c455-pksb8"
Jul 17 04:51:04.999771 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:04.999335    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a143a72-1103-45f5-97b1-18bcb51c06bd-console-serving-cert\") pod \"console-75ff58c455-pksb8\" (UID: \"5a143a72-1103-45f5-97b1-18bcb51c06bd\") " pod="openshift-console/console-75ff58c455-pksb8"
Jul 17 04:51:04.999771 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:04.999362    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a143a72-1103-45f5-97b1-18bcb51c06bd-service-ca\") pod \"console-75ff58c455-pksb8\" (UID: \"5a143a72-1103-45f5-97b1-18bcb51c06bd\") " pod="openshift-console/console-75ff58c455-pksb8"
Jul 17 04:51:04.999771 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:04.999377    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a143a72-1103-45f5-97b1-18bcb51c06bd-oauth-serving-cert\") pod \"console-75ff58c455-pksb8\" (UID: \"5a143a72-1103-45f5-97b1-18bcb51c06bd\") " pod="openshift-console/console-75ff58c455-pksb8"
Jul 17 04:51:04.999771 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:04.999426    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a143a72-1103-45f5-97b1-18bcb51c06bd-console-oauth-config\") pod \"console-75ff58c455-pksb8\" (UID: \"5a143a72-1103-45f5-97b1-18bcb51c06bd\") " pod="openshift-console/console-75ff58c455-pksb8"
Jul 17 04:51:05.100548 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:05.100508    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a143a72-1103-45f5-97b1-18bcb51c06bd-console-oauth-config\") pod \"console-75ff58c455-pksb8\" (UID: \"5a143a72-1103-45f5-97b1-18bcb51c06bd\") " pod="openshift-console/console-75ff58c455-pksb8"
Jul 17 04:51:05.100726 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:05.100565    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a143a72-1103-45f5-97b1-18bcb51c06bd-console-config\") pod \"console-75ff58c455-pksb8\" (UID: \"5a143a72-1103-45f5-97b1-18bcb51c06bd\") " pod="openshift-console/console-75ff58c455-pksb8"
Jul 17 04:51:05.100726 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:05.100595    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njq58\" (UniqueName: \"kubernetes.io/projected/5a143a72-1103-45f5-97b1-18bcb51c06bd-kube-api-access-njq58\") pod \"console-75ff58c455-pksb8\" (UID: \"5a143a72-1103-45f5-97b1-18bcb51c06bd\") " pod="openshift-console/console-75ff58c455-pksb8"
Jul 17 04:51:05.100726 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:05.100631    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a143a72-1103-45f5-97b1-18bcb51c06bd-trusted-ca-bundle\") pod \"console-75ff58c455-pksb8\" (UID: \"5a143a72-1103-45f5-97b1-18bcb51c06bd\") " pod="openshift-console/console-75ff58c455-pksb8"
Jul 17 04:51:05.100726 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:05.100655    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a143a72-1103-45f5-97b1-18bcb51c06bd-console-serving-cert\") pod \"console-75ff58c455-pksb8\" (UID: \"5a143a72-1103-45f5-97b1-18bcb51c06bd\") " pod="openshift-console/console-75ff58c455-pksb8"
Jul 17 04:51:05.100726 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:05.100689    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a143a72-1103-45f5-97b1-18bcb51c06bd-service-ca\") pod \"console-75ff58c455-pksb8\" (UID: \"5a143a72-1103-45f5-97b1-18bcb51c06bd\") " pod="openshift-console/console-75ff58c455-pksb8"
Jul 17 04:51:05.100726 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:05.100710    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a143a72-1103-45f5-97b1-18bcb51c06bd-oauth-serving-cert\") pod \"console-75ff58c455-pksb8\" (UID: \"5a143a72-1103-45f5-97b1-18bcb51c06bd\") " pod="openshift-console/console-75ff58c455-pksb8"
Jul 17 04:51:05.101764 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:05.101362    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a143a72-1103-45f5-97b1-18bcb51c06bd-oauth-serving-cert\") pod \"console-75ff58c455-pksb8\" (UID: \"5a143a72-1103-45f5-97b1-18bcb51c06bd\") " pod="openshift-console/console-75ff58c455-pksb8"
Jul 17 04:51:05.101764 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:05.101436    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a143a72-1103-45f5-97b1-18bcb51c06bd-console-config\") pod \"console-75ff58c455-pksb8\" (UID: \"5a143a72-1103-45f5-97b1-18bcb51c06bd\") " pod="openshift-console/console-75ff58c455-pksb8"
Jul 17 04:51:05.101764 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:05.101700    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a143a72-1103-45f5-97b1-18bcb51c06bd-service-ca\") pod \"console-75ff58c455-pksb8\" (UID: \"5a143a72-1103-45f5-97b1-18bcb51c06bd\") " pod="openshift-console/console-75ff58c455-pksb8"
Jul 17 04:51:05.101987 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:05.101936    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a143a72-1103-45f5-97b1-18bcb51c06bd-trusted-ca-bundle\") pod \"console-75ff58c455-pksb8\" (UID: \"5a143a72-1103-45f5-97b1-18bcb51c06bd\") " pod="openshift-console/console-75ff58c455-pksb8"
Jul 17 04:51:05.105327 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:05.105302    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a143a72-1103-45f5-97b1-18bcb51c06bd-console-serving-cert\") pod \"console-75ff58c455-pksb8\" (UID: \"5a143a72-1103-45f5-97b1-18bcb51c06bd\") " pod="openshift-console/console-75ff58c455-pksb8"
Jul 17 04:51:05.106004 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:05.105956    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a143a72-1103-45f5-97b1-18bcb51c06bd-console-oauth-config\") pod \"console-75ff58c455-pksb8\" (UID: \"5a143a72-1103-45f5-97b1-18bcb51c06bd\") " pod="openshift-console/console-75ff58c455-pksb8"
Jul 17 04:51:05.114310 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:05.114265    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njq58\" (UniqueName: \"kubernetes.io/projected/5a143a72-1103-45f5-97b1-18bcb51c06bd-kube-api-access-njq58\") pod \"console-75ff58c455-pksb8\" (UID: \"5a143a72-1103-45f5-97b1-18bcb51c06bd\") " pod="openshift-console/console-75ff58c455-pksb8"
Jul 17 04:51:05.135038 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:05.135001    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75ff58c455-pksb8"
Jul 17 04:51:05.498093 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:05.498055    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75ff58c455-pksb8"]
Jul 17 04:51:05.512083 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:05.512050    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-kxf2j"]
Jul 17 04:51:05.596465 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:51:05.596430    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a143a72_1103_45f5_97b1_18bcb51c06bd.slice/crio-2fc47a04a80baac9088ca91e8132d1214796f0524b1ca2b34761b39f3e8ccd5a WatchSource:0}: Error finding container 2fc47a04a80baac9088ca91e8132d1214796f0524b1ca2b34761b39f3e8ccd5a: Status 404 returned error can't find the container with id 2fc47a04a80baac9088ca91e8132d1214796f0524b1ca2b34761b39f3e8ccd5a
Jul 17 04:51:05.597072 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:51:05.597050    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87aa6ee8_0708_428a_985b_40dbd41f57cb.slice/crio-cfd6c8c30db895b4cf37023724175242562987bc4a2376795117763fd1157be0 WatchSource:0}: Error finding container cfd6c8c30db895b4cf37023724175242562987bc4a2376795117763fd1157be0: Status 404 returned error can't find the container with id cfd6c8c30db895b4cf37023724175242562987bc4a2376795117763fd1157be0
Jul 17 04:51:06.171526 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:06.171490    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-kxf2j" event={"ID":"87aa6ee8-0708-428a-985b-40dbd41f57cb","Type":"ContainerStarted","Data":"cfd6c8c30db895b4cf37023724175242562987bc4a2376795117763fd1157be0"}
Jul 17 04:51:06.173327 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:06.173296    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kf2nj" event={"ID":"9b4a2877-135f-4be6-a249-83c2006db349","Type":"ContainerStarted","Data":"9e7bd999cda13ba15cfd57b0d4b1a6ccb1cc636be3502dd17c8f435eb639e0c4"}
Jul 17 04:51:06.175106 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:06.175074    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75ff58c455-pksb8" event={"ID":"5a143a72-1103-45f5-97b1-18bcb51c06bd","Type":"ContainerStarted","Data":"2fc47a04a80baac9088ca91e8132d1214796f0524b1ca2b34761b39f3e8ccd5a"}
Jul 17 04:51:10.190696 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:10.189029    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pwfdn" event={"ID":"5ddbfecb-13e8-4c64-942b-6b8100d6b92a","Type":"ContainerStarted","Data":"8b8c88b9f5a8d8a1a67fbeb108fb034f718ade868649a13e7a5f4f9d2e2f068e"}
Jul 17 04:51:10.191513 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:10.191486    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8pbjc" event={"ID":"3f3f79dc-2048-413f-a701-1ac6a22554a4","Type":"ContainerStarted","Data":"f1772a3338c4d0ac5990c6dcb888d8206796d51ee1a4b7766cd9ea749288a76b"}
Jul 17 04:51:10.193936 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:10.193909    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-kxf2j" event={"ID":"87aa6ee8-0708-428a-985b-40dbd41f57cb","Type":"ContainerStarted","Data":"fafa9d156134a6ceb3917402f43b6344fd4d7f4ea781dfc54529f17dba4f72ae"}
Jul 17 04:51:10.194122 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:10.194102    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-kxf2j"
Jul 17 04:51:10.196787 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:10.196756    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kf2nj" event={"ID":"9b4a2877-135f-4be6-a249-83c2006db349","Type":"ContainerStarted","Data":"24b80f615814c8d0b389ecc13698d2c628a0544b99871a8d2e36e5a8ecb3b44b"}
Jul 17 04:51:10.196876 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:10.196852    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-kf2nj"
Jul 17 04:51:10.200191 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:10.200173    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-kxf2j"
Jul 17 04:51:10.212137 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:10.212091    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-8pbjc" podStartSLOduration=2.090428006 podStartE2EDuration="8.212075916s" totalImagesPullingTime="6.12164791s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 04:51:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 04:51:10.210854052 +0000 UTC m=+57.894447709" watchObservedRunningTime="2026-07-17 04:51:10.212075916 +0000 UTC m=+57.895669572"
Jul 17 04:51:10.235181 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:10.235124    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-kf2nj" podStartSLOduration=5.834374557 podStartE2EDuration="8.23510722s" totalImagesPullingTime="2.400732663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 04:51:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 04:51:10.234127325 +0000 UTC m=+57.917721007" watchObservedRunningTime="2026-07-17 04:51:10.23510722 +0000 UTC m=+57.918700878"
Jul 17 04:51:10.253037 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:10.252952    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-kxf2j" podStartSLOduration=3.188056884 podStartE2EDuration="7.252933031s" totalImagesPullingTime="4.064876147s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 04:51:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 04:51:10.252538869 +0000 UTC m=+57.936132555" watchObservedRunningTime="2026-07-17 04:51:10.252933031 +0000 UTC m=+57.936526689"
Jul 17 04:51:10.719613 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:10.719582    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-nqj6x"]
Jul 17 04:51:10.725115 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:10.725093    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-nqj6x"
Jul 17 04:51:10.729067 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:10.729040    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 17 04:51:10.729209 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:10.729078    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 17 04:51:10.729209 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:10.729094    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-q7fcn\""
Jul 17 04:51:10.729209 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:10.729148    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 17 04:51:10.729209 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:10.729040    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 17 04:51:10.729472 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:10.729450    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 17 04:51:10.733729 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:10.733438    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-nqj6x"]
Jul 17 04:51:10.745512 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:10.745484    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/49508546-6acb-4970-a9ae-25fad03768fd-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-nqj6x\" (UID: \"49508546-6acb-4970-a9ae-25fad03768fd\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-nqj6x"
Jul 17 04:51:10.745646 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:10.745554    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/49508546-6acb-4970-a9ae-25fad03768fd-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-nqj6x\" (UID: \"49508546-6acb-4970-a9ae-25fad03768fd\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-nqj6x"
Jul 17 04:51:10.745646 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:10.745600    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/49508546-6acb-4970-a9ae-25fad03768fd-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-nqj6x\" (UID: \"49508546-6acb-4970-a9ae-25fad03768fd\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-nqj6x"
Jul 17 04:51:10.745646 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:10.745628    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbn94\" (UniqueName: \"kubernetes.io/projected/49508546-6acb-4970-a9ae-25fad03768fd-kube-api-access-zbn94\") pod \"prometheus-operator-5945b5cfd-nqj6x\" (UID: \"49508546-6acb-4970-a9ae-25fad03768fd\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-nqj6x"
Jul 17 04:51:10.846546 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:10.846513    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/49508546-6acb-4970-a9ae-25fad03768fd-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-nqj6x\" (UID: \"49508546-6acb-4970-a9ae-25fad03768fd\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-nqj6x"
Jul 17 04:51:10.846753 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:10.846576    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/49508546-6acb-4970-a9ae-25fad03768fd-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-nqj6x\" (UID: \"49508546-6acb-4970-a9ae-25fad03768fd\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-nqj6x"
Jul 17 04:51:10.846753 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:10.846612    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbn94\" (UniqueName: \"kubernetes.io/projected/49508546-6acb-4970-a9ae-25fad03768fd-kube-api-access-zbn94\") pod \"prometheus-operator-5945b5cfd-nqj6x\" (UID: \"49508546-6acb-4970-a9ae-25fad03768fd\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-nqj6x"
Jul 17 04:51:10.846878 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:10.846790    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/49508546-6acb-4970-a9ae-25fad03768fd-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-nqj6x\" (UID: \"49508546-6acb-4970-a9ae-25fad03768fd\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-nqj6x"
Jul 17 04:51:10.847500 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:10.847467    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/49508546-6acb-4970-a9ae-25fad03768fd-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-nqj6x\" (UID: \"49508546-6acb-4970-a9ae-25fad03768fd\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-nqj6x"
Jul 17 04:51:10.849992 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:10.849945    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/49508546-6acb-4970-a9ae-25fad03768fd-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-nqj6x\" (UID: \"49508546-6acb-4970-a9ae-25fad03768fd\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-nqj6x"
Jul 17 04:51:10.850179 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:10.850155    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/49508546-6acb-4970-a9ae-25fad03768fd-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-nqj6x\" (UID: \"49508546-6acb-4970-a9ae-25fad03768fd\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-nqj6x"
Jul 17 04:51:10.855461 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:10.855443    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbn94\" (UniqueName: \"kubernetes.io/projected/49508546-6acb-4970-a9ae-25fad03768fd-kube-api-access-zbn94\") pod \"prometheus-operator-5945b5cfd-nqj6x\" (UID: \"49508546-6acb-4970-a9ae-25fad03768fd\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-nqj6x"
Jul 17 04:51:11.036893 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:11.036868    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-nqj6x"
Jul 17 04:51:11.203948 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:11.203860    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75ff58c455-pksb8" event={"ID":"5a143a72-1103-45f5-97b1-18bcb51c06bd","Type":"ContainerStarted","Data":"924c1a8a1e0595154ed9677a1d93d8b62fc78391d4b327b7b6d4daabf22e35bb"}
Jul 17 04:51:11.207664 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:11.207573    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-nqj6x"]
Jul 17 04:51:11.211124 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:51:11.210799    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49508546_6acb_4970_a9ae_25fad03768fd.slice/crio-d565af97434f258c14ca61ea779bc0f1aae56cce20ab1fee6b7a457e3e1a9b15 WatchSource:0}: Error finding container d565af97434f258c14ca61ea779bc0f1aae56cce20ab1fee6b7a457e3e1a9b15: Status 404 returned error can't find the container with id d565af97434f258c14ca61ea779bc0f1aae56cce20ab1fee6b7a457e3e1a9b15
Jul 17 04:51:11.227339 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:11.227283    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-75ff58c455-pksb8" podStartSLOduration=1.835135027 podStartE2EDuration="7.227262457s" totalImagesPullingTime="5.39212743s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 04:51:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 04:51:11.226529954 +0000 UTC m=+58.910123614" watchObservedRunningTime="2026-07-17 04:51:11.227262457 +0000 UTC m=+58.910856103"
Jul 17 04:51:12.210379 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:12.210339    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-nqj6x" event={"ID":"49508546-6acb-4970-a9ae-25fad03768fd","Type":"ContainerStarted","Data":"d565af97434f258c14ca61ea779bc0f1aae56cce20ab1fee6b7a457e3e1a9b15"}
Jul 17 04:51:13.113265 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:13.113236    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tx9gj"
Jul 17 04:51:15.135324 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:15.135266    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-75ff58c455-pksb8"
Jul 17 04:51:15.135740 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:15.135356    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-75ff58c455-pksb8"
Jul 17 04:51:15.140124 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:15.140100    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-75ff58c455-pksb8"
Jul 17 04:51:15.223474 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:15.223421    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-nqj6x" event={"ID":"49508546-6acb-4970-a9ae-25fad03768fd","Type":"ContainerStarted","Data":"71675a88e8066c395fe1a2bc96a7b2b259119ad6f95a486f84dceef36c482684"}
Jul 17 04:51:15.223651 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:15.223491    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-nqj6x" event={"ID":"49508546-6acb-4970-a9ae-25fad03768fd","Type":"ContainerStarted","Data":"9f8f8e966c9295811ef160957cc815260088a0b7145ae9f42489bab8da536e1d"}
Jul 17 04:51:15.225782 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:15.225753    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pwfdn" event={"ID":"5ddbfecb-13e8-4c64-942b-6b8100d6b92a","Type":"ContainerStarted","Data":"415961efe3d0390ffebe335e622fcf6423e58c7318810ecbf662a89411f47831"}
Jul 17 04:51:15.231215 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:15.231190    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-75ff58c455-pksb8"
Jul 17 04:51:15.240248 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:15.240197    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-nqj6x" podStartSLOduration=2.010808711 podStartE2EDuration="5.240182787s" totalImagesPullingTime="3.229374076s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 04:51:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 04:51:15.239262977 +0000 UTC m=+62.922856635" watchObservedRunningTime="2026-07-17 04:51:15.240182787 +0000 UTC m=+62.923776444"
Jul 17 04:51:17.068566 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.068530    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-8td74"]
Jul 17 04:51:17.081664 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.081637    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8td74"
Jul 17 04:51:17.084010 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.083954    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-8td74"]
Jul 17 04:51:17.084946 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.084922    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 17 04:51:17.084946 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.084941    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-596mw\""
Jul 17 04:51:17.085418 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.085130    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 17 04:51:17.094031 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.094010    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-hb8td"]
Jul 17 04:51:17.097305 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.097282    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-hb8td"
Jul 17 04:51:17.100681 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.100084    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-q56nx\""
Jul 17 04:51:17.100681 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.100144    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 17 04:51:17.100681 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.100155    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 17 04:51:17.100681 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.100367    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 17 04:51:17.101536 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.101515    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/07582083-1a89-49cd-904e-09a6df8d62fe-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-8td74\" (UID: \"07582083-1a89-49cd-904e-09a6df8d62fe\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8td74"
Jul 17 04:51:17.101634 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.101565    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/07582083-1a89-49cd-904e-09a6df8d62fe-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-8td74\" (UID: \"07582083-1a89-49cd-904e-09a6df8d62fe\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8td74"
Jul 17 04:51:17.101716 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.101692    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cm59w\" (UniqueName: \"kubernetes.io/projected/07582083-1a89-49cd-904e-09a6df8d62fe-kube-api-access-cm59w\") pod \"openshift-state-metrics-79f6958cdf-8td74\" (UID: \"07582083-1a89-49cd-904e-09a6df8d62fe\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8td74"
Jul 17 04:51:17.101755 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.101741    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/07582083-1a89-49cd-904e-09a6df8d62fe-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-8td74\" (UID: \"07582083-1a89-49cd-904e-09a6df8d62fe\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8td74"
Jul 17 04:51:17.108064 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.108026    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-hb8td"]
Jul 17 04:51:17.111040 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.111018    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-jtngp"]
Jul 17 04:51:17.114680 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.114659    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jtngp"
Jul 17 04:51:17.116915 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.116831    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 17 04:51:17.117210 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.117173    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-2j6t8\""
Jul 17 04:51:17.117377 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.117363    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 17 04:51:17.117426 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.117363    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 17 04:51:17.202145 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.202106    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a16c8d78-1778-44b1-acb1-f8e609e3b331-root\") pod \"node-exporter-jtngp\" (UID: \"a16c8d78-1778-44b1-acb1-f8e609e3b331\") " pod="openshift-monitoring/node-exporter-jtngp"
Jul 17 04:51:17.202145 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.202144    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a16c8d78-1778-44b1-acb1-f8e609e3b331-node-exporter-tls\") pod \"node-exporter-jtngp\" (UID: \"a16c8d78-1778-44b1-acb1-f8e609e3b331\") " pod="openshift-monitoring/node-exporter-jtngp"
Jul 17 04:51:17.202350 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.202170    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a16c8d78-1778-44b1-acb1-f8e609e3b331-node-exporter-accelerators-collector-config\") pod \"node-exporter-jtngp\" (UID: \"a16c8d78-1778-44b1-acb1-f8e609e3b331\") " pod="openshift-monitoring/node-exporter-jtngp"
Jul 17 04:51:17.202350 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.202208    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d8bf6d68-c828-41db-995e-6484b8cbf930-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-hb8td\" (UID: \"d8bf6d68-c828-41db-995e-6484b8cbf930\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-hb8td"
Jul 17 04:51:17.202350 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.202224    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbwtn\" (UniqueName: \"kubernetes.io/projected/a16c8d78-1778-44b1-acb1-f8e609e3b331-kube-api-access-sbwtn\") pod \"node-exporter-jtngp\" (UID: \"a16c8d78-1778-44b1-acb1-f8e609e3b331\") " pod="openshift-monitoring/node-exporter-jtngp"
Jul 17 04:51:17.202350 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.202259    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a16c8d78-1778-44b1-acb1-f8e609e3b331-metrics-client-ca\") pod \"node-exporter-jtngp\" (UID: \"a16c8d78-1778-44b1-acb1-f8e609e3b331\") " pod="openshift-monitoring/node-exporter-jtngp"
Jul 17 04:51:17.202350 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.202310    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cm59w\" (UniqueName: \"kubernetes.io/projected/07582083-1a89-49cd-904e-09a6df8d62fe-kube-api-access-cm59w\") pod \"openshift-state-metrics-79f6958cdf-8td74\" (UID: \"07582083-1a89-49cd-904e-09a6df8d62fe\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8td74"
Jul 17 04:51:17.202554 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.202351    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a16c8d78-1778-44b1-acb1-f8e609e3b331-node-exporter-wtmp\") pod \"node-exporter-jtngp\" (UID: \"a16c8d78-1778-44b1-acb1-f8e609e3b331\") " pod="openshift-monitoring/node-exporter-jtngp"
Jul 17 04:51:17.202554 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.202377    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cppk8\" (UniqueName: \"kubernetes.io/projected/d8bf6d68-c828-41db-995e-6484b8cbf930-kube-api-access-cppk8\") pod \"kube-state-metrics-56c5594674-hb8td\" (UID: \"d8bf6d68-c828-41db-995e-6484b8cbf930\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-hb8td"
Jul 17 04:51:17.202554 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.202404    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/07582083-1a89-49cd-904e-09a6df8d62fe-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-8td74\" (UID: \"07582083-1a89-49cd-904e-09a6df8d62fe\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8td74"
Jul 17 04:51:17.202554 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.202441    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/07582083-1a89-49cd-904e-09a6df8d62fe-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-8td74\" (UID: \"07582083-1a89-49cd-904e-09a6df8d62fe\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8td74"
Jul 17 04:51:17.202554 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.202461    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d8bf6d68-c828-41db-995e-6484b8cbf930-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-hb8td\" (UID: \"d8bf6d68-c828-41db-995e-6484b8cbf930\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-hb8td"
Jul 17 04:51:17.202554 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.202479    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d8bf6d68-c828-41db-995e-6484b8cbf930-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-hb8td\" (UID: \"d8bf6d68-c828-41db-995e-6484b8cbf930\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-hb8td"
Jul 17 04:51:17.202554 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.202507    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a16c8d78-1778-44b1-acb1-f8e609e3b331-node-exporter-textfile\") pod \"node-exporter-jtngp\" (UID: \"a16c8d78-1778-44b1-acb1-f8e609e3b331\") " pod="openshift-monitoring/node-exporter-jtngp"
Jul 17 04:51:17.202554 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.202528    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/07582083-1a89-49cd-904e-09a6df8d62fe-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-8td74\" (UID: \"07582083-1a89-49cd-904e-09a6df8d62fe\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8td74"
Jul 17 04:51:17.202554 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.202556    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d8bf6d68-c828-41db-995e-6484b8cbf930-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-hb8td\" (UID: \"d8bf6d68-c828-41db-995e-6484b8cbf930\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-hb8td"
Jul 17 04:51:17.202892 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:51:17.202564    2562 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 17 04:51:17.202892 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.202604    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d8bf6d68-c828-41db-995e-6484b8cbf930-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-hb8td\" (UID: \"d8bf6d68-c828-41db-995e-6484b8cbf930\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-hb8td"
Jul 17 04:51:17.202892 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:51:17.202640    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07582083-1a89-49cd-904e-09a6df8d62fe-openshift-state-metrics-tls podName:07582083-1a89-49cd-904e-09a6df8d62fe nodeName:}" failed. No retries permitted until 2026-07-17 04:51:17.702619812 +0000 UTC m=+65.386213453 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/07582083-1a89-49cd-904e-09a6df8d62fe-openshift-state-metrics-tls") pod "openshift-state-metrics-79f6958cdf-8td74" (UID: "07582083-1a89-49cd-904e-09a6df8d62fe") : secret "openshift-state-metrics-tls" not found
Jul 17 04:51:17.202892 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.202665    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a16c8d78-1778-44b1-acb1-f8e609e3b331-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jtngp\" (UID: \"a16c8d78-1778-44b1-acb1-f8e609e3b331\") " pod="openshift-monitoring/node-exporter-jtngp"
Jul 17 04:51:17.202892 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.202699    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a16c8d78-1778-44b1-acb1-f8e609e3b331-sys\") pod \"node-exporter-jtngp\" (UID: \"a16c8d78-1778-44b1-acb1-f8e609e3b331\") " pod="openshift-monitoring/node-exporter-jtngp"
Jul 17 04:51:17.203126 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.203092    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/07582083-1a89-49cd-904e-09a6df8d62fe-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-8td74\" (UID: \"07582083-1a89-49cd-904e-09a6df8d62fe\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8td74"
Jul 17 04:51:17.205428 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.205398    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/07582083-1a89-49cd-904e-09a6df8d62fe-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-8td74\" (UID: \"07582083-1a89-49cd-904e-09a6df8d62fe\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8td74"
Jul 17 04:51:17.207175 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.207157    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-kf2nj"
Jul 17 04:51:17.213461 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.213440    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cm59w\" (UniqueName: \"kubernetes.io/projected/07582083-1a89-49cd-904e-09a6df8d62fe-kube-api-access-cm59w\") pod \"openshift-state-metrics-79f6958cdf-8td74\" (UID: \"07582083-1a89-49cd-904e-09a6df8d62fe\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8td74"
Jul 17 04:51:17.303571 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.303538    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d8bf6d68-c828-41db-995e-6484b8cbf930-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-hb8td\" (UID: \"d8bf6d68-c828-41db-995e-6484b8cbf930\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-hb8td"
Jul 17 04:51:17.303752 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.303576    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d8bf6d68-c828-41db-995e-6484b8cbf930-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-hb8td\" (UID: \"d8bf6d68-c828-41db-995e-6484b8cbf930\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-hb8td"
Jul 17 04:51:17.303752 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:51:17.303713    2562 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 17 04:51:17.303876 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:51:17.303792    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8bf6d68-c828-41db-995e-6484b8cbf930-kube-state-metrics-tls podName:d8bf6d68-c828-41db-995e-6484b8cbf930 nodeName:}" failed. No retries permitted until 2026-07-17 04:51:17.80377163 +0000 UTC m=+65.487365278 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/d8bf6d68-c828-41db-995e-6484b8cbf930-kube-state-metrics-tls") pod "kube-state-metrics-56c5594674-hb8td" (UID: "d8bf6d68-c828-41db-995e-6484b8cbf930") : secret "kube-state-metrics-tls" not found
Jul 17 04:51:17.304057 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.304026    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a16c8d78-1778-44b1-acb1-f8e609e3b331-node-exporter-textfile\") pod \"node-exporter-jtngp\" (UID: \"a16c8d78-1778-44b1-acb1-f8e609e3b331\") " pod="openshift-monitoring/node-exporter-jtngp"
Jul 17 04:51:17.304181 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.304102    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d8bf6d68-c828-41db-995e-6484b8cbf930-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-hb8td\" (UID: \"d8bf6d68-c828-41db-995e-6484b8cbf930\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-hb8td"
Jul 17 04:51:17.304244 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.304190    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d8bf6d68-c828-41db-995e-6484b8cbf930-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-hb8td\" (UID: \"d8bf6d68-c828-41db-995e-6484b8cbf930\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-hb8td"
Jul 17 04:51:17.304244 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.304218    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a16c8d78-1778-44b1-acb1-f8e609e3b331-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jtngp\" (UID: \"a16c8d78-1778-44b1-acb1-f8e609e3b331\") " pod="openshift-monitoring/node-exporter-jtngp"
Jul 17 04:51:17.304349 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.304245    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a16c8d78-1778-44b1-acb1-f8e609e3b331-sys\") pod \"node-exporter-jtngp\" (UID: \"a16c8d78-1778-44b1-acb1-f8e609e3b331\") " pod="openshift-monitoring/node-exporter-jtngp"
Jul 17 04:51:17.304349 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.304271    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a16c8d78-1778-44b1-acb1-f8e609e3b331-root\") pod \"node-exporter-jtngp\" (UID: \"a16c8d78-1778-44b1-acb1-f8e609e3b331\") " pod="openshift-monitoring/node-exporter-jtngp"
Jul 17 04:51:17.304349 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.304285    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d8bf6d68-c828-41db-995e-6484b8cbf930-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-hb8td\" (UID: \"d8bf6d68-c828-41db-995e-6484b8cbf930\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-hb8td"
Jul 17 04:51:17.304349 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.304301    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a16c8d78-1778-44b1-acb1-f8e609e3b331-node-exporter-tls\") pod \"node-exporter-jtngp\" (UID: \"a16c8d78-1778-44b1-acb1-f8e609e3b331\") " pod="openshift-monitoring/node-exporter-jtngp"
Jul 17 04:51:17.304349 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.304326    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a16c8d78-1778-44b1-acb1-f8e609e3b331-node-exporter-accelerators-collector-config\") pod \"node-exporter-jtngp\" (UID: \"a16c8d78-1778-44b1-acb1-f8e609e3b331\") " pod="openshift-monitoring/node-exporter-jtngp"
Jul 17 04:51:17.304349 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.304327    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a16c8d78-1778-44b1-acb1-f8e609e3b331-node-exporter-textfile\") pod \"node-exporter-jtngp\" (UID: \"a16c8d78-1778-44b1-acb1-f8e609e3b331\") " pod="openshift-monitoring/node-exporter-jtngp"
Jul 17 04:51:17.304630 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.304366    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d8bf6d68-c828-41db-995e-6484b8cbf930-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-hb8td\" (UID: \"d8bf6d68-c828-41db-995e-6484b8cbf930\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-hb8td"
Jul 17 04:51:17.304630 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.304392    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbwtn\" (UniqueName: \"kubernetes.io/projected/a16c8d78-1778-44b1-acb1-f8e609e3b331-kube-api-access-sbwtn\") pod \"node-exporter-jtngp\" (UID: \"a16c8d78-1778-44b1-acb1-f8e609e3b331\") " pod="openshift-monitoring/node-exporter-jtngp"
Jul 17 04:51:17.304630 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.304405    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a16c8d78-1778-44b1-acb1-f8e609e3b331-root\") pod \"node-exporter-jtngp\" (UID: \"a16c8d78-1778-44b1-acb1-f8e609e3b331\") " pod="openshift-monitoring/node-exporter-jtngp"
Jul 17 04:51:17.304796 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.304601    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a16c8d78-1778-44b1-acb1-f8e609e3b331-metrics-client-ca\") pod \"node-exporter-jtngp\" (UID: \"a16c8d78-1778-44b1-acb1-f8e609e3b331\") " pod="openshift-monitoring/node-exporter-jtngp"
Jul 17 04:51:17.304796 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.304690    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a16c8d78-1778-44b1-acb1-f8e609e3b331-node-exporter-wtmp\") pod \"node-exporter-jtngp\" (UID: \"a16c8d78-1778-44b1-acb1-f8e609e3b331\") " pod="openshift-monitoring/node-exporter-jtngp"
Jul 17 04:51:17.304796 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.304730    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cppk8\" (UniqueName: \"kubernetes.io/projected/d8bf6d68-c828-41db-995e-6484b8cbf930-kube-api-access-cppk8\") pod \"kube-state-metrics-56c5594674-hb8td\" (UID: \"d8bf6d68-c828-41db-995e-6484b8cbf930\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-hb8td"
Jul 17 04:51:17.305181 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.305114    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a16c8d78-1778-44b1-acb1-f8e609e3b331-node-exporter-accelerators-collector-config\") pod \"node-exporter-jtngp\" (UID: \"a16c8d78-1778-44b1-acb1-f8e609e3b331\") " pod="openshift-monitoring/node-exporter-jtngp"
Jul 17 04:51:17.305276 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:51:17.305200    2562 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 17 04:51:17.305276 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.305210    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d8bf6d68-c828-41db-995e-6484b8cbf930-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-hb8td\" (UID: \"d8bf6d68-c828-41db-995e-6484b8cbf930\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-hb8td"
Jul 17 04:51:17.305276 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:51:17.305243    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a16c8d78-1778-44b1-acb1-f8e609e3b331-node-exporter-tls podName:a16c8d78-1778-44b1-acb1-f8e609e3b331 nodeName:}" failed. No retries permitted until 2026-07-17 04:51:17.805226026 +0000 UTC m=+65.488819660 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/a16c8d78-1778-44b1-acb1-f8e609e3b331-node-exporter-tls") pod "node-exporter-jtngp" (UID: "a16c8d78-1778-44b1-acb1-f8e609e3b331") : secret "node-exporter-tls" not found
Jul 17 04:51:17.305276 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.305210    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a16c8d78-1778-44b1-acb1-f8e609e3b331-sys\") pod \"node-exporter-jtngp\" (UID: \"a16c8d78-1778-44b1-acb1-f8e609e3b331\") " pod="openshift-monitoring/node-exporter-jtngp"
Jul 17 04:51:17.305601 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.305380    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a16c8d78-1778-44b1-acb1-f8e609e3b331-node-exporter-wtmp\") pod \"node-exporter-jtngp\" (UID: \"a16c8d78-1778-44b1-acb1-f8e609e3b331\") " pod="openshift-monitoring/node-exporter-jtngp"
Jul 17 04:51:17.305826 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.305797    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d8bf6d68-c828-41db-995e-6484b8cbf930-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-hb8td\" (UID: \"d8bf6d68-c828-41db-995e-6484b8cbf930\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-hb8td"
Jul 17 04:51:17.306033 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.306010    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a16c8d78-1778-44b1-acb1-f8e609e3b331-metrics-client-ca\") pod \"node-exporter-jtngp\" (UID: \"a16c8d78-1778-44b1-acb1-f8e609e3b331\") " pod="openshift-monitoring/node-exporter-jtngp"
Jul 17 04:51:17.307202 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.307182    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a16c8d78-1778-44b1-acb1-f8e609e3b331-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jtngp\" (UID: \"a16c8d78-1778-44b1-acb1-f8e609e3b331\") " pod="openshift-monitoring/node-exporter-jtngp"
Jul 17 04:51:17.307294 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.307268    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d8bf6d68-c828-41db-995e-6484b8cbf930-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-hb8td\" (UID: \"d8bf6d68-c828-41db-995e-6484b8cbf930\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-hb8td"
Jul 17 04:51:17.316721 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.316698    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbwtn\" (UniqueName: \"kubernetes.io/projected/a16c8d78-1778-44b1-acb1-f8e609e3b331-kube-api-access-sbwtn\") pod \"node-exporter-jtngp\" (UID: \"a16c8d78-1778-44b1-acb1-f8e609e3b331\") " pod="openshift-monitoring/node-exporter-jtngp"
Jul 17 04:51:17.316721 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.316711    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cppk8\" (UniqueName: \"kubernetes.io/projected/d8bf6d68-c828-41db-995e-6484b8cbf930-kube-api-access-cppk8\") pod \"kube-state-metrics-56c5594674-hb8td\" (UID: \"d8bf6d68-c828-41db-995e-6484b8cbf930\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-hb8td"
Jul 17 04:51:17.710250 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.710208    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/07582083-1a89-49cd-904e-09a6df8d62fe-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-8td74\" (UID: \"07582083-1a89-49cd-904e-09a6df8d62fe\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8td74"
Jul 17 04:51:17.713040 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.713011    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/07582083-1a89-49cd-904e-09a6df8d62fe-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-8td74\" (UID: \"07582083-1a89-49cd-904e-09a6df8d62fe\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8td74"
Jul 17 04:51:17.811405 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.811363    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a16c8d78-1778-44b1-acb1-f8e609e3b331-node-exporter-tls\") pod \"node-exporter-jtngp\" (UID: \"a16c8d78-1778-44b1-acb1-f8e609e3b331\") " pod="openshift-monitoring/node-exporter-jtngp"
Jul 17 04:51:17.811575 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.811472    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d8bf6d68-c828-41db-995e-6484b8cbf930-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-hb8td\" (UID: \"d8bf6d68-c828-41db-995e-6484b8cbf930\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-hb8td"
Jul 17 04:51:17.811575 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:51:17.811523    2562 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 17 04:51:17.811673 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:51:17.811608    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a16c8d78-1778-44b1-acb1-f8e609e3b331-node-exporter-tls podName:a16c8d78-1778-44b1-acb1-f8e609e3b331 nodeName:}" failed. No retries permitted until 2026-07-17 04:51:18.81158646 +0000 UTC m=+66.495180096 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/a16c8d78-1778-44b1-acb1-f8e609e3b331-node-exporter-tls") pod "node-exporter-jtngp" (UID: "a16c8d78-1778-44b1-acb1-f8e609e3b331") : secret "node-exporter-tls" not found
Jul 17 04:51:17.814419 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.814393    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d8bf6d68-c828-41db-995e-6484b8cbf930-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-hb8td\" (UID: \"d8bf6d68-c828-41db-995e-6484b8cbf930\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-hb8td"
Jul 17 04:51:17.993592 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:17.993495    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8td74"
Jul 17 04:51:18.007832 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.007796    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-hb8td"
Jul 17 04:51:18.165881 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.165818    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 17 04:51:18.172296 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.171700    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 17 04:51:18.176738 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.176327    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 17 04:51:18.176738 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.176525    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 17 04:51:18.176738 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.176576    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 17 04:51:18.180004 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.178384    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 17 04:51:18.180004 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.178649    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 17 04:51:18.180004 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.179281    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 17 04:51:18.180004 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.179484    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 17 04:51:18.180004 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.179750    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 17 04:51:18.180384 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.180070    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 17 04:51:18.180443 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.180432    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-lfpdj\""
Jul 17 04:51:18.185037 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.184987    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 17 04:51:18.220190 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.216878    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2975ade8-a4ab-4bec-9781-64655a501d21-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2975ade8-a4ab-4bec-9781-64655a501d21\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 04:51:18.220190 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.216938    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2975ade8-a4ab-4bec-9781-64655a501d21-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2975ade8-a4ab-4bec-9781-64655a501d21\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 04:51:18.220190 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.217027    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2975ade8-a4ab-4bec-9781-64655a501d21-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2975ade8-a4ab-4bec-9781-64655a501d21\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 04:51:18.220190 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.217134    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2975ade8-a4ab-4bec-9781-64655a501d21-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2975ade8-a4ab-4bec-9781-64655a501d21\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 04:51:18.220190 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.217175    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2975ade8-a4ab-4bec-9781-64655a501d21-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2975ade8-a4ab-4bec-9781-64655a501d21\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 04:51:18.220190 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.217207    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2975ade8-a4ab-4bec-9781-64655a501d21-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2975ade8-a4ab-4bec-9781-64655a501d21\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 04:51:18.220190 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.217238    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2975ade8-a4ab-4bec-9781-64655a501d21-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2975ade8-a4ab-4bec-9781-64655a501d21\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 04:51:18.220190 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.217261    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2975ade8-a4ab-4bec-9781-64655a501d21-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2975ade8-a4ab-4bec-9781-64655a501d21\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 04:51:18.220190 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.217302    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bf75j\" (UniqueName: \"kubernetes.io/projected/2975ade8-a4ab-4bec-9781-64655a501d21-kube-api-access-bf75j\") pod \"alertmanager-main-0\" (UID: \"2975ade8-a4ab-4bec-9781-64655a501d21\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 04:51:18.220190 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.217346    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2975ade8-a4ab-4bec-9781-64655a501d21-config-out\") pod \"alertmanager-main-0\" (UID: \"2975ade8-a4ab-4bec-9781-64655a501d21\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 04:51:18.220190 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.217390    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2975ade8-a4ab-4bec-9781-64655a501d21-config-volume\") pod \"alertmanager-main-0\" (UID: \"2975ade8-a4ab-4bec-9781-64655a501d21\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 04:51:18.220190 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.217414    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2975ade8-a4ab-4bec-9781-64655a501d21-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2975ade8-a4ab-4bec-9781-64655a501d21\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 04:51:18.220190 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.217462    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2975ade8-a4ab-4bec-9781-64655a501d21-web-config\") pod \"alertmanager-main-0\" (UID: \"2975ade8-a4ab-4bec-9781-64655a501d21\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 04:51:18.277430 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.277388    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-hb8td"]
Jul 17 04:51:18.281149 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:51:18.281125    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8bf6d68_c828_41db_995e_6484b8cbf930.slice/crio-d787f70deefb3b3060d8762050c2669f09df541ebc39dcc9861d0a06c0ce1396 WatchSource:0}: Error finding container d787f70deefb3b3060d8762050c2669f09df541ebc39dcc9861d0a06c0ce1396: Status 404 returned error can't find the container with id d787f70deefb3b3060d8762050c2669f09df541ebc39dcc9861d0a06c0ce1396
Jul 17 04:51:18.295067 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.294931    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-8td74"]
Jul 17 04:51:18.297504 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:51:18.297477    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07582083_1a89_49cd_904e_09a6df8d62fe.slice/crio-0e989bf0a73fff2a570b555095ec391784295f5d7261e19bce3f13fe58b1cc94 WatchSource:0}: Error finding container 0e989bf0a73fff2a570b555095ec391784295f5d7261e19bce3f13fe58b1cc94: Status 404 returned error can't find the container with id 0e989bf0a73fff2a570b555095ec391784295f5d7261e19bce3f13fe58b1cc94
Jul 17 04:51:18.318006 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.317974    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2975ade8-a4ab-4bec-9781-64655a501d21-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2975ade8-a4ab-4bec-9781-64655a501d21\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 04:51:18.318159 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.318011    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2975ade8-a4ab-4bec-9781-64655a501d21-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2975ade8-a4ab-4bec-9781-64655a501d21\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 04:51:18.318159 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.318032    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2975ade8-a4ab-4bec-9781-64655a501d21-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2975ade8-a4ab-4bec-9781-64655a501d21\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 04:51:18.318159 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.318057    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bf75j\" (UniqueName: \"kubernetes.io/projected/2975ade8-a4ab-4bec-9781-64655a501d21-kube-api-access-bf75j\") pod \"alertmanager-main-0\" (UID: \"2975ade8-a4ab-4bec-9781-64655a501d21\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 04:51:18.318159 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.318091    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2975ade8-a4ab-4bec-9781-64655a501d21-config-out\") pod \"alertmanager-main-0\" (UID: \"2975ade8-a4ab-4bec-9781-64655a501d21\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 04:51:18.318159 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.318125    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2975ade8-a4ab-4bec-9781-64655a501d21-config-volume\") pod \"alertmanager-main-0\" (UID: \"2975ade8-a4ab-4bec-9781-64655a501d21\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 04:51:18.318392 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.318254    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2975ade8-a4ab-4bec-9781-64655a501d21-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2975ade8-a4ab-4bec-9781-64655a501d21\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 04:51:18.318392 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.318328    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2975ade8-a4ab-4bec-9781-64655a501d21-web-config\") pod \"alertmanager-main-0\" (UID: \"2975ade8-a4ab-4bec-9781-64655a501d21\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 04:51:18.318392 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.318360    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2975ade8-a4ab-4bec-9781-64655a501d21-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2975ade8-a4ab-4bec-9781-64655a501d21\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 04:51:18.318545 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.318388    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2975ade8-a4ab-4bec-9781-64655a501d21-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2975ade8-a4ab-4bec-9781-64655a501d21\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 04:51:18.318545 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.318430    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2975ade8-a4ab-4bec-9781-64655a501d21-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2975ade8-a4ab-4bec-9781-64655a501d21\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 04:51:18.319281 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.319212    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2975ade8-a4ab-4bec-9781-64655a501d21-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2975ade8-a4ab-4bec-9781-64655a501d21\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 04:51:18.320435 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.320404    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2975ade8-a4ab-4bec-9781-64655a501d21-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2975ade8-a4ab-4bec-9781-64655a501d21\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 04:51:18.320555 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.320472    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2975ade8-a4ab-4bec-9781-64655a501d21-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2975ade8-a4ab-4bec-9781-64655a501d21\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 04:51:18.320555 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.320522    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2975ade8-a4ab-4bec-9781-64655a501d21-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2975ade8-a4ab-4bec-9781-64655a501d21\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 04:51:18.320751 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:51:18.320726    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2975ade8-a4ab-4bec-9781-64655a501d21-alertmanager-trusted-ca-bundle podName:2975ade8-a4ab-4bec-9781-64655a501d21 nodeName:}" failed. No retries permitted until 2026-07-17 04:51:18.820644543 +0000 UTC m=+66.504238186 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/2975ade8-a4ab-4bec-9781-64655a501d21-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "2975ade8-a4ab-4bec-9781-64655a501d21") : configmap references non-existent config key: ca-bundle.crt
Jul 17 04:51:18.321626 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.321602    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2975ade8-a4ab-4bec-9781-64655a501d21-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2975ade8-a4ab-4bec-9781-64655a501d21\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 04:51:18.321734 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.321641    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2975ade8-a4ab-4bec-9781-64655a501d21-web-config\") pod \"alertmanager-main-0\" (UID: \"2975ade8-a4ab-4bec-9781-64655a501d21\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 04:51:18.321873 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.321847    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2975ade8-a4ab-4bec-9781-64655a501d21-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2975ade8-a4ab-4bec-9781-64655a501d21\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 04:51:18.322168 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.322132    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2975ade8-a4ab-4bec-9781-64655a501d21-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2975ade8-a4ab-4bec-9781-64655a501d21\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 04:51:18.322301 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.322167    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2975ade8-a4ab-4bec-9781-64655a501d21-config-volume\") pod \"alertmanager-main-0\" (UID: \"2975ade8-a4ab-4bec-9781-64655a501d21\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 04:51:18.322634 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.322583    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2975ade8-a4ab-4bec-9781-64655a501d21-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2975ade8-a4ab-4bec-9781-64655a501d21\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 04:51:18.322736 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.322650    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2975ade8-a4ab-4bec-9781-64655a501d21-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2975ade8-a4ab-4bec-9781-64655a501d21\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 04:51:18.323034 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.323014    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2975ade8-a4ab-4bec-9781-64655a501d21-config-out\") pod \"alertmanager-main-0\" (UID: \"2975ade8-a4ab-4bec-9781-64655a501d21\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 04:51:18.323904 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.323882    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2975ade8-a4ab-4bec-9781-64655a501d21-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2975ade8-a4ab-4bec-9781-64655a501d21\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 04:51:18.326616 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.326593    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bf75j\" (UniqueName: \"kubernetes.io/projected/2975ade8-a4ab-4bec-9781-64655a501d21-kube-api-access-bf75j\") pod \"alertmanager-main-0\" (UID: \"2975ade8-a4ab-4bec-9781-64655a501d21\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 04:51:18.522177 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.522101    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a54b09be-6889-45d6-a748-68e5b94b73d2-metrics-certs\") pod \"network-metrics-daemon-rzdct\" (UID: \"a54b09be-6889-45d6-a748-68e5b94b73d2\") " pod="openshift-multus/network-metrics-daemon-rzdct"
Jul 17 04:51:18.524788 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.524770    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 17 04:51:18.535235 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.535213    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a54b09be-6889-45d6-a748-68e5b94b73d2-metrics-certs\") pod \"network-metrics-daemon-rzdct\" (UID: \"a54b09be-6889-45d6-a748-68e5b94b73d2\") " pod="openshift-multus/network-metrics-daemon-rzdct"
Jul 17 04:51:18.609626 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.609598    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zwbtr\""
Jul 17 04:51:18.617786 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.617763    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzdct"
Jul 17 04:51:18.623229 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.623205    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4xwm\" (UniqueName: \"kubernetes.io/projected/bcee811d-1474-45f6-b0c7-afceee510ccd-kube-api-access-l4xwm\") pod \"network-check-target-9vmqd\" (UID: \"bcee811d-1474-45f6-b0c7-afceee510ccd\") " pod="openshift-network-diagnostics/network-check-target-9vmqd"
Jul 17 04:51:18.623318 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.623276    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/237abccc-c19c-4fca-be94-d3013f3c2e86-original-pull-secret\") pod \"global-pull-secret-syncer-6bkt5\" (UID: \"237abccc-c19c-4fca-be94-d3013f3c2e86\") " pod="kube-system/global-pull-secret-syncer-6bkt5"
Jul 17 04:51:18.625749 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.625731    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 17 04:51:18.625894 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.625878    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 17 04:51:18.636019 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.636001    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 17 04:51:18.636221 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.636205    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/237abccc-c19c-4fca-be94-d3013f3c2e86-original-pull-secret\") pod \"global-pull-secret-syncer-6bkt5\" (UID: \"237abccc-c19c-4fca-be94-d3013f3c2e86\") " pod="kube-system/global-pull-secret-syncer-6bkt5"
Jul 17 04:51:18.646481 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.646458    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4xwm\" (UniqueName: \"kubernetes.io/projected/bcee811d-1474-45f6-b0c7-afceee510ccd-kube-api-access-l4xwm\") pod \"network-check-target-9vmqd\" (UID: \"bcee811d-1474-45f6-b0c7-afceee510ccd\") " pod="openshift-network-diagnostics/network-check-target-9vmqd"
Jul 17 04:51:18.762349 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.762233    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rzdct"]
Jul 17 04:51:18.765076 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:51:18.764945    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda54b09be_6889_45d6_a748_68e5b94b73d2.slice/crio-24b766e309dc7ea52dcf71aae8abcd37d34a0327b9552a9ac143ae25be296926 WatchSource:0}: Error finding container 24b766e309dc7ea52dcf71aae8abcd37d34a0327b9552a9ac143ae25be296926: Status 404 returned error can't find the container with id 24b766e309dc7ea52dcf71aae8abcd37d34a0327b9552a9ac143ae25be296926
Jul 17 04:51:18.825119 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.825090    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a16c8d78-1778-44b1-acb1-f8e609e3b331-node-exporter-tls\") pod \"node-exporter-jtngp\" (UID: \"a16c8d78-1778-44b1-acb1-f8e609e3b331\") " pod="openshift-monitoring/node-exporter-jtngp"
Jul 17 04:51:18.825288 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.825128    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2975ade8-a4ab-4bec-9781-64655a501d21-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2975ade8-a4ab-4bec-9781-64655a501d21\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 04:51:18.826024 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.825994    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2975ade8-a4ab-4bec-9781-64655a501d21-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2975ade8-a4ab-4bec-9781-64655a501d21\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 04:51:18.827580 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.827556    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a16c8d78-1778-44b1-acb1-f8e609e3b331-node-exporter-tls\") pod \"node-exporter-jtngp\" (UID: \"a16c8d78-1778-44b1-acb1-f8e609e3b331\") " pod="openshift-monitoring/node-exporter-jtngp"
Jul 17 04:51:18.893089 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.893056    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6bkt5"
Jul 17 04:51:18.903892 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.903865    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mhn7j\""
Jul 17 04:51:18.911383 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.911358    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9vmqd"
Jul 17 04:51:18.923877 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:18.923853    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jtngp"
Jul 17 04:51:18.980402 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:51:18.980367    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda16c8d78_1778_44b1_acb1_f8e609e3b331.slice/crio-df902a28b07f0b40ccff906c713be80c244b47b1d6828dee084c9754296b9390 WatchSource:0}: Error finding container df902a28b07f0b40ccff906c713be80c244b47b1d6828dee084c9754296b9390: Status 404 returned error can't find the container with id df902a28b07f0b40ccff906c713be80c244b47b1d6828dee084c9754296b9390
Jul 17 04:51:19.068781 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.068729    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6bkt5"]
Jul 17 04:51:19.072150 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:51:19.072120    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod237abccc_c19c_4fca_be94_d3013f3c2e86.slice/crio-3a8e5922a012dff129e19657b331a6653f34a27c24df4474a4ba4fccbc399b81 WatchSource:0}: Error finding container 3a8e5922a012dff129e19657b331a6653f34a27c24df4474a4ba4fccbc399b81: Status 404 returned error can't find the container with id 3a8e5922a012dff129e19657b331a6653f34a27c24df4474a4ba4fccbc399b81
Jul 17 04:51:19.078045 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.078022    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7875fbb676-p8ln5"]
Jul 17 04:51:19.096678 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.096628    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 17 04:51:19.109301 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.109256    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7875fbb676-p8ln5"]
Jul 17 04:51:19.109479 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.109368    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7875fbb676-p8ln5"
Jul 17 04:51:19.111980 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.111944    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9vmqd"]
Jul 17 04:51:19.112686 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.112664    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 17 04:51:19.112758 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.112665    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 17 04:51:19.112803 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.112765    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-qcgkr\""
Jul 17 04:51:19.113414 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.113224    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 17 04:51:19.113414 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.113351    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-b3q57j43pjt2f\""
Jul 17 04:51:19.113629 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.113479    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 17 04:51:19.113629 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.113590    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 17 04:51:19.114332 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:51:19.114307    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcee811d_1474_45f6_b0c7_afceee510ccd.slice/crio-6e9cde05bcd32835247e11a206fa0f2b40f5011b3c0b89a8b3eab86338f21d3d WatchSource:0}: Error finding container 6e9cde05bcd32835247e11a206fa0f2b40f5011b3c0b89a8b3eab86338f21d3d: Status 404 returned error can't find the container with id 6e9cde05bcd32835247e11a206fa0f2b40f5011b3c0b89a8b3eab86338f21d3d
Jul 17 04:51:19.230044 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.229636    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/679b67c5-405f-4561-9846-8aaef06ee7bc-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7875fbb676-p8ln5\" (UID: \"679b67c5-405f-4561-9846-8aaef06ee7bc\") " pod="openshift-monitoring/thanos-querier-7875fbb676-p8ln5"
Jul 17 04:51:19.230044 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.229672    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/679b67c5-405f-4561-9846-8aaef06ee7bc-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7875fbb676-p8ln5\" (UID: \"679b67c5-405f-4561-9846-8aaef06ee7bc\") " pod="openshift-monitoring/thanos-querier-7875fbb676-p8ln5"
Jul 17 04:51:19.230044 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.229712    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/679b67c5-405f-4561-9846-8aaef06ee7bc-secret-grpc-tls\") pod \"thanos-querier-7875fbb676-p8ln5\" (UID: \"679b67c5-405f-4561-9846-8aaef06ee7bc\") " pod="openshift-monitoring/thanos-querier-7875fbb676-p8ln5"
Jul 17 04:51:19.230044 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.229752    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/679b67c5-405f-4561-9846-8aaef06ee7bc-metrics-client-ca\") pod \"thanos-querier-7875fbb676-p8ln5\" (UID: \"679b67c5-405f-4561-9846-8aaef06ee7bc\") " pod="openshift-monitoring/thanos-querier-7875fbb676-p8ln5"
Jul 17 04:51:19.230044 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.229777    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvs2w\" (UniqueName: \"kubernetes.io/projected/679b67c5-405f-4561-9846-8aaef06ee7bc-kube-api-access-qvs2w\") pod \"thanos-querier-7875fbb676-p8ln5\" (UID: \"679b67c5-405f-4561-9846-8aaef06ee7bc\") " pod="openshift-monitoring/thanos-querier-7875fbb676-p8ln5"
Jul 17 04:51:19.230044 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.229836    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/679b67c5-405f-4561-9846-8aaef06ee7bc-secret-thanos-querier-tls\") pod \"thanos-querier-7875fbb676-p8ln5\" (UID: \"679b67c5-405f-4561-9846-8aaef06ee7bc\") " pod="openshift-monitoring/thanos-querier-7875fbb676-p8ln5"
Jul 17 04:51:19.230044 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.229864    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/679b67c5-405f-4561-9846-8aaef06ee7bc-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7875fbb676-p8ln5\" (UID: \"679b67c5-405f-4561-9846-8aaef06ee7bc\") " pod="openshift-monitoring/thanos-querier-7875fbb676-p8ln5"
Jul 17 04:51:19.230044 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.229912    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/679b67c5-405f-4561-9846-8aaef06ee7bc-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7875fbb676-p8ln5\" (UID: \"679b67c5-405f-4561-9846-8aaef06ee7bc\") " pod="openshift-monitoring/thanos-querier-7875fbb676-p8ln5"
Jul 17 04:51:19.247009 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.246943    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jtngp" event={"ID":"a16c8d78-1778-44b1-acb1-f8e609e3b331","Type":"ContainerStarted","Data":"df902a28b07f0b40ccff906c713be80c244b47b1d6828dee084c9754296b9390"}
Jul 17 04:51:19.248546 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.248509    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6bkt5" event={"ID":"237abccc-c19c-4fca-be94-d3013f3c2e86","Type":"ContainerStarted","Data":"3a8e5922a012dff129e19657b331a6653f34a27c24df4474a4ba4fccbc399b81"}
Jul 17 04:51:19.250510 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.250457    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rzdct" event={"ID":"a54b09be-6889-45d6-a748-68e5b94b73d2","Type":"ContainerStarted","Data":"24b766e309dc7ea52dcf71aae8abcd37d34a0327b9552a9ac143ae25be296926"}
Jul 17 04:51:19.252164 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.252135    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-hb8td" event={"ID":"d8bf6d68-c828-41db-995e-6484b8cbf930","Type":"ContainerStarted","Data":"d787f70deefb3b3060d8762050c2669f09df541ebc39dcc9861d0a06c0ce1396"}
Jul 17 04:51:19.254547 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.254521    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8td74" event={"ID":"07582083-1a89-49cd-904e-09a6df8d62fe","Type":"ContainerStarted","Data":"b3dc52f3b1bfd2452f0e77ab35e508fa0827e17df058f29e4750575a21ca90ec"}
Jul 17 04:51:19.254675 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.254659    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8td74" event={"ID":"07582083-1a89-49cd-904e-09a6df8d62fe","Type":"ContainerStarted","Data":"6c345649f760ef105e1fd94155dc6b0a3dedca7420fcb4acc2be57c684a05ca8"}
Jul 17 04:51:19.254759 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.254744    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8td74" event={"ID":"07582083-1a89-49cd-904e-09a6df8d62fe","Type":"ContainerStarted","Data":"0e989bf0a73fff2a570b555095ec391784295f5d7261e19bce3f13fe58b1cc94"}
Jul 17 04:51:19.258142 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.258114    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pwfdn" event={"ID":"5ddbfecb-13e8-4c64-942b-6b8100d6b92a","Type":"ContainerStarted","Data":"c0397768b90a05e7f87a63e5e93a730cb2a277607b38c64d79db3c9e73c25d47"}
Jul 17 04:51:19.260592 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.260568    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9vmqd" event={"ID":"bcee811d-1474-45f6-b0c7-afceee510ccd","Type":"ContainerStarted","Data":"6e9cde05bcd32835247e11a206fa0f2b40f5011b3c0b89a8b3eab86338f21d3d"}
Jul 17 04:51:19.280930 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.279181    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-pwfdn" podStartSLOduration=2.52473502 podStartE2EDuration="17.279163975s" totalImagesPullingTime="14.754428955s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 04:51:02 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 04:51:19.278834784 +0000 UTC m=+66.962428461" watchObservedRunningTime="2026-07-17 04:51:19.279163975 +0000 UTC m=+66.962757611"
Jul 17 04:51:19.290921 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.290795    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 17 04:51:19.293143 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:51:19.293112    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2975ade8_a4ab_4bec_9781_64655a501d21.slice/crio-be1e62b42c986ffcd6d9a17fe89a24b011e5cb74d2cb467422012c9bc413e18f WatchSource:0}: Error finding container be1e62b42c986ffcd6d9a17fe89a24b011e5cb74d2cb467422012c9bc413e18f: Status 404 returned error can't find the container with id be1e62b42c986ffcd6d9a17fe89a24b011e5cb74d2cb467422012c9bc413e18f
Jul 17 04:51:19.331110 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.330616    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/679b67c5-405f-4561-9846-8aaef06ee7bc-secret-grpc-tls\") pod \"thanos-querier-7875fbb676-p8ln5\" (UID: \"679b67c5-405f-4561-9846-8aaef06ee7bc\") " pod="openshift-monitoring/thanos-querier-7875fbb676-p8ln5"
Jul 17 04:51:19.331110 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.330904    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/679b67c5-405f-4561-9846-8aaef06ee7bc-metrics-client-ca\") pod \"thanos-querier-7875fbb676-p8ln5\" (UID: \"679b67c5-405f-4561-9846-8aaef06ee7bc\") " pod="openshift-monitoring/thanos-querier-7875fbb676-p8ln5"
Jul 17 04:51:19.331110 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.330950    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvs2w\" (UniqueName: \"kubernetes.io/projected/679b67c5-405f-4561-9846-8aaef06ee7bc-kube-api-access-qvs2w\") pod \"thanos-querier-7875fbb676-p8ln5\" (UID: \"679b67c5-405f-4561-9846-8aaef06ee7bc\") " pod="openshift-monitoring/thanos-querier-7875fbb676-p8ln5"
Jul 17 04:51:19.331110 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.331034    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/679b67c5-405f-4561-9846-8aaef06ee7bc-secret-thanos-querier-tls\") pod \"thanos-querier-7875fbb676-p8ln5\" (UID: \"679b67c5-405f-4561-9846-8aaef06ee7bc\") " pod="openshift-monitoring/thanos-querier-7875fbb676-p8ln5"
Jul 17 04:51:19.331110 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.331064    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/679b67c5-405f-4561-9846-8aaef06ee7bc-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7875fbb676-p8ln5\" (UID: \"679b67c5-405f-4561-9846-8aaef06ee7bc\") " pod="openshift-monitoring/thanos-querier-7875fbb676-p8ln5"
Jul 17 04:51:19.331470 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.331119    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/679b67c5-405f-4561-9846-8aaef06ee7bc-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7875fbb676-p8ln5\" (UID: \"679b67c5-405f-4561-9846-8aaef06ee7bc\") " pod="openshift-monitoring/thanos-querier-7875fbb676-p8ln5"
Jul 17 04:51:19.331470 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.331166    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/679b67c5-405f-4561-9846-8aaef06ee7bc-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7875fbb676-p8ln5\" (UID: \"679b67c5-405f-4561-9846-8aaef06ee7bc\") " pod="openshift-monitoring/thanos-querier-7875fbb676-p8ln5"
Jul 17 04:51:19.331470 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.331215    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/679b67c5-405f-4561-9846-8aaef06ee7bc-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7875fbb676-p8ln5\" (UID: \"679b67c5-405f-4561-9846-8aaef06ee7bc\") " pod="openshift-monitoring/thanos-querier-7875fbb676-p8ln5"
Jul 17 04:51:19.332406 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.331610    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/679b67c5-405f-4561-9846-8aaef06ee7bc-metrics-client-ca\") pod \"thanos-querier-7875fbb676-p8ln5\" (UID: \"679b67c5-405f-4561-9846-8aaef06ee7bc\") " pod="openshift-monitoring/thanos-querier-7875fbb676-p8ln5"
Jul 17 04:51:19.334047 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.333980    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/679b67c5-405f-4561-9846-8aaef06ee7bc-secret-grpc-tls\") pod \"thanos-querier-7875fbb676-p8ln5\" (UID: \"679b67c5-405f-4561-9846-8aaef06ee7bc\") " pod="openshift-monitoring/thanos-querier-7875fbb676-p8ln5"
Jul 17 04:51:19.338017 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.337990    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/679b67c5-405f-4561-9846-8aaef06ee7bc-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7875fbb676-p8ln5\" (UID: \"679b67c5-405f-4561-9846-8aaef06ee7bc\") " pod="openshift-monitoring/thanos-querier-7875fbb676-p8ln5"
Jul 17 04:51:19.338523 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.338496    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/679b67c5-405f-4561-9846-8aaef06ee7bc-secret-thanos-querier-tls\") pod \"thanos-querier-7875fbb676-p8ln5\" (UID: \"679b67c5-405f-4561-9846-8aaef06ee7bc\") " pod="openshift-monitoring/thanos-querier-7875fbb676-p8ln5"
Jul 17 04:51:19.338934 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.338911    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/679b67c5-405f-4561-9846-8aaef06ee7bc-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7875fbb676-p8ln5\" (UID: \"679b67c5-405f-4561-9846-8aaef06ee7bc\") " pod="openshift-monitoring/thanos-querier-7875fbb676-p8ln5"
Jul 17 04:51:19.339368 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.339307    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/679b67c5-405f-4561-9846-8aaef06ee7bc-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7875fbb676-p8ln5\" (UID: \"679b67c5-405f-4561-9846-8aaef06ee7bc\") " pod="openshift-monitoring/thanos-querier-7875fbb676-p8ln5"
Jul 17 04:51:19.339750 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.339725    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/679b67c5-405f-4561-9846-8aaef06ee7bc-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7875fbb676-p8ln5\" (UID: \"679b67c5-405f-4561-9846-8aaef06ee7bc\") " pod="openshift-monitoring/thanos-querier-7875fbb676-p8ln5"
Jul 17 04:51:19.340646 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.340621    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvs2w\" (UniqueName: \"kubernetes.io/projected/679b67c5-405f-4561-9846-8aaef06ee7bc-kube-api-access-qvs2w\") pod \"thanos-querier-7875fbb676-p8ln5\" (UID: \"679b67c5-405f-4561-9846-8aaef06ee7bc\") " pod="openshift-monitoring/thanos-querier-7875fbb676-p8ln5"
Jul 17 04:51:19.421202 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.421158    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7875fbb676-p8ln5"
Jul 17 04:51:19.597477 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:19.597337    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7875fbb676-p8ln5"]
Jul 17 04:51:20.073437 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:51:20.073408    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod679b67c5_405f_4561_9846_8aaef06ee7bc.slice/crio-3bce330331d7acf239368bb68ed947db7425b2573ff8168282cd4bae65b92458 WatchSource:0}: Error finding container 3bce330331d7acf239368bb68ed947db7425b2573ff8168282cd4bae65b92458: Status 404 returned error can't find the container with id 3bce330331d7acf239368bb68ed947db7425b2573ff8168282cd4bae65b92458
Jul 17 04:51:20.267006 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:20.266937    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2975ade8-a4ab-4bec-9781-64655a501d21","Type":"ContainerStarted","Data":"be1e62b42c986ffcd6d9a17fe89a24b011e5cb74d2cb467422012c9bc413e18f"}
Jul 17 04:51:20.268356 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:20.268326    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7875fbb676-p8ln5" event={"ID":"679b67c5-405f-4561-9846-8aaef06ee7bc","Type":"ContainerStarted","Data":"3bce330331d7acf239368bb68ed947db7425b2573ff8168282cd4bae65b92458"}
Jul 17 04:51:21.279431 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.279258    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8td74" event={"ID":"07582083-1a89-49cd-904e-09a6df8d62fe","Type":"ContainerStarted","Data":"7c5231a53d6a6ad193dc122e4ac4bbe60fb48648a9fab8b7869d6169f0d15a02"}
Jul 17 04:51:21.285024 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.284773    2562 generic.go:356] "Generic (PLEG): container finished" podID="a16c8d78-1778-44b1-acb1-f8e609e3b331" containerID="058f9751fe04bce3d37276dff31d26f99c2c6f8851f02aca0bc97e4d301c7443" exitCode=0
Jul 17 04:51:21.285024 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.284923    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jtngp" event={"ID":"a16c8d78-1778-44b1-acb1-f8e609e3b331","Type":"ContainerDied","Data":"058f9751fe04bce3d37276dff31d26f99c2c6f8851f02aca0bc97e4d301c7443"}
Jul 17 04:51:21.292146 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.291554    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rzdct" event={"ID":"a54b09be-6889-45d6-a748-68e5b94b73d2","Type":"ContainerStarted","Data":"1e9613747cfe529bd33ff94989c0fa7c28b55a5936d4ee588ff1dc1c22aa4105"}
Jul 17 04:51:21.292146 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.291609    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rzdct" event={"ID":"a54b09be-6889-45d6-a748-68e5b94b73d2","Type":"ContainerStarted","Data":"1c94bd5df7b7aa5dc801801e6973fc2a8c749bc858d4be4c03ca2e03b8d40a0d"}
Jul 17 04:51:21.296943 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.295510    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-hb8td" event={"ID":"d8bf6d68-c828-41db-995e-6484b8cbf930","Type":"ContainerStarted","Data":"3b67d0e8c9f318bca08d3270993dca4f34c7bdffbc86900df346a8a63d203bb5"}
Jul 17 04:51:21.296943 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.295569    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-hb8td" event={"ID":"d8bf6d68-c828-41db-995e-6484b8cbf930","Type":"ContainerStarted","Data":"4ac2b41ab4e37e8960152f61d2fa4844be9b27224321d780a0cefe97bdc1345a"}
Jul 17 04:51:21.296943 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.295599    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-hb8td" event={"ID":"d8bf6d68-c828-41db-995e-6484b8cbf930","Type":"ContainerStarted","Data":"5ab9d9154ebf5693297bdcec5e1939eb0c1db89f99c1c3ddbb7799c4e019b469"}
Jul 17 04:51:21.302506 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.301420    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8td74" podStartSLOduration=2.263541554 podStartE2EDuration="4.301400634s" totalImagesPullingTime="2.03785908s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 04:51:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 04:51:21.300413909 +0000 UTC m=+68.984007604" watchObservedRunningTime="2026-07-17 04:51:21.301400634 +0000 UTC m=+68.984994292"
Jul 17 04:51:21.325234 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.324345    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-rzdct" podStartSLOduration=67.485337027 podStartE2EDuration="1m9.32432416s" totalImagesPullingTime="1.838987133s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 04:50:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 04:51:21.323238883 +0000 UTC m=+69.006832566" watchObservedRunningTime="2026-07-17 04:51:21.32432416 +0000 UTC m=+69.007917819"
Jul 17 04:51:21.397895 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.396445    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-hb8td" podStartSLOduration=2.075128684 podStartE2EDuration="4.396423558s" totalImagesPullingTime="2.321294874s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 04:51:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 04:51:21.357742092 +0000 UTC m=+69.041335767" watchObservedRunningTime="2026-07-17 04:51:21.396423558 +0000 UTC m=+69.080017215"
Jul 17 04:51:21.458842 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.457256    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-64f677f8cd-7x5kz"]
Jul 17 04:51:21.461156 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.460344    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-64f677f8cd-7x5kz"
Jul 17 04:51:21.462914 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.462875    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-2ees2f6accbet\""
Jul 17 04:51:21.463370 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.463116    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 17 04:51:21.463370 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.463255    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 17 04:51:21.463507 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.463371    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-bnljn\""
Jul 17 04:51:21.463665 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.463646    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 17 04:51:21.463956 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.463921    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 17 04:51:21.471271 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.471243    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-64f677f8cd-7x5kz"]
Jul 17 04:51:21.551132 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.551085    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7ff86f69-6040-4003-89d8-84ed6d165084-secret-metrics-server-tls\") pod \"metrics-server-64f677f8cd-7x5kz\" (UID: \"7ff86f69-6040-4003-89d8-84ed6d165084\") " pod="openshift-monitoring/metrics-server-64f677f8cd-7x5kz"
Jul 17 04:51:21.551328 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.551157    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7ff86f69-6040-4003-89d8-84ed6d165084-metrics-server-audit-profiles\") pod \"metrics-server-64f677f8cd-7x5kz\" (UID: \"7ff86f69-6040-4003-89d8-84ed6d165084\") " pod="openshift-monitoring/metrics-server-64f677f8cd-7x5kz"
Jul 17 04:51:21.551328 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.551181    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7ff86f69-6040-4003-89d8-84ed6d165084-client-ca-bundle\") pod \"metrics-server-64f677f8cd-7x5kz\" (UID: \"7ff86f69-6040-4003-89d8-84ed6d165084\") " pod="openshift-monitoring/metrics-server-64f677f8cd-7x5kz"
Jul 17 04:51:21.551328 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.551198    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7ff86f69-6040-4003-89d8-84ed6d165084-audit-log\") pod \"metrics-server-64f677f8cd-7x5kz\" (UID: \"7ff86f69-6040-4003-89d8-84ed6d165084\") " pod="openshift-monitoring/metrics-server-64f677f8cd-7x5kz"
Jul 17 04:51:21.551328 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.551225    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ff86f69-6040-4003-89d8-84ed6d165084-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-64f677f8cd-7x5kz\" (UID: \"7ff86f69-6040-4003-89d8-84ed6d165084\") " pod="openshift-monitoring/metrics-server-64f677f8cd-7x5kz"
Jul 17 04:51:21.551328 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.551253    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4ffh\" (UniqueName: \"kubernetes.io/projected/7ff86f69-6040-4003-89d8-84ed6d165084-kube-api-access-g4ffh\") pod \"metrics-server-64f677f8cd-7x5kz\" (UID: \"7ff86f69-6040-4003-89d8-84ed6d165084\") " pod="openshift-monitoring/metrics-server-64f677f8cd-7x5kz"
Jul 17 04:51:21.551328 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.551271    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7ff86f69-6040-4003-89d8-84ed6d165084-secret-metrics-server-client-certs\") pod \"metrics-server-64f677f8cd-7x5kz\" (UID: \"7ff86f69-6040-4003-89d8-84ed6d165084\") " pod="openshift-monitoring/metrics-server-64f677f8cd-7x5kz"
Jul 17 04:51:21.652798 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.652248    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7ff86f69-6040-4003-89d8-84ed6d165084-secret-metrics-server-tls\") pod \"metrics-server-64f677f8cd-7x5kz\" (UID: \"7ff86f69-6040-4003-89d8-84ed6d165084\") " pod="openshift-monitoring/metrics-server-64f677f8cd-7x5kz"
Jul 17 04:51:21.652798 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.652342    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7ff86f69-6040-4003-89d8-84ed6d165084-metrics-server-audit-profiles\") pod \"metrics-server-64f677f8cd-7x5kz\" (UID: \"7ff86f69-6040-4003-89d8-84ed6d165084\") " pod="openshift-monitoring/metrics-server-64f677f8cd-7x5kz"
Jul 17 04:51:21.652798 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.652371    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7ff86f69-6040-4003-89d8-84ed6d165084-client-ca-bundle\") pod \"metrics-server-64f677f8cd-7x5kz\" (UID: \"7ff86f69-6040-4003-89d8-84ed6d165084\") " pod="openshift-monitoring/metrics-server-64f677f8cd-7x5kz"
Jul 17 04:51:21.652798 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.652397    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7ff86f69-6040-4003-89d8-84ed6d165084-audit-log\") pod \"metrics-server-64f677f8cd-7x5kz\" (UID: \"7ff86f69-6040-4003-89d8-84ed6d165084\") " pod="openshift-monitoring/metrics-server-64f677f8cd-7x5kz"
Jul 17 04:51:21.652798 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.652428    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ff86f69-6040-4003-89d8-84ed6d165084-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-64f677f8cd-7x5kz\" (UID: \"7ff86f69-6040-4003-89d8-84ed6d165084\") " pod="openshift-monitoring/metrics-server-64f677f8cd-7x5kz"
Jul 17 04:51:21.652798 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.652455    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4ffh\" (UniqueName: \"kubernetes.io/projected/7ff86f69-6040-4003-89d8-84ed6d165084-kube-api-access-g4ffh\") pod \"metrics-server-64f677f8cd-7x5kz\" (UID: \"7ff86f69-6040-4003-89d8-84ed6d165084\") " pod="openshift-monitoring/metrics-server-64f677f8cd-7x5kz"
Jul 17 04:51:21.652798 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.652481    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7ff86f69-6040-4003-89d8-84ed6d165084-secret-metrics-server-client-certs\") pod \"metrics-server-64f677f8cd-7x5kz\" (UID: \"7ff86f69-6040-4003-89d8-84ed6d165084\") " pod="openshift-monitoring/metrics-server-64f677f8cd-7x5kz"
Jul 17 04:51:21.654641 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.654143    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7ff86f69-6040-4003-89d8-84ed6d165084-metrics-server-audit-profiles\") pod \"metrics-server-64f677f8cd-7x5kz\" (UID: \"7ff86f69-6040-4003-89d8-84ed6d165084\") " pod="openshift-monitoring/metrics-server-64f677f8cd-7x5kz"
Jul 17 04:51:21.654641 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.654241    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7ff86f69-6040-4003-89d8-84ed6d165084-audit-log\") pod \"metrics-server-64f677f8cd-7x5kz\" (UID: \"7ff86f69-6040-4003-89d8-84ed6d165084\") " pod="openshift-monitoring/metrics-server-64f677f8cd-7x5kz"
Jul 17 04:51:21.654641 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.654573    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ff86f69-6040-4003-89d8-84ed6d165084-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-64f677f8cd-7x5kz\" (UID: \"7ff86f69-6040-4003-89d8-84ed6d165084\") " pod="openshift-monitoring/metrics-server-64f677f8cd-7x5kz"
Jul 17 04:51:21.656665 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.656638    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7ff86f69-6040-4003-89d8-84ed6d165084-secret-metrics-server-tls\") pod \"metrics-server-64f677f8cd-7x5kz\" (UID: \"7ff86f69-6040-4003-89d8-84ed6d165084\") " pod="openshift-monitoring/metrics-server-64f677f8cd-7x5kz"
Jul 17 04:51:21.657366 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.657342    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7ff86f69-6040-4003-89d8-84ed6d165084-secret-metrics-server-client-certs\") pod \"metrics-server-64f677f8cd-7x5kz\" (UID: \"7ff86f69-6040-4003-89d8-84ed6d165084\") " pod="openshift-monitoring/metrics-server-64f677f8cd-7x5kz"
Jul 17 04:51:21.659063 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.659014    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7ff86f69-6040-4003-89d8-84ed6d165084-client-ca-bundle\") pod \"metrics-server-64f677f8cd-7x5kz\" (UID: \"7ff86f69-6040-4003-89d8-84ed6d165084\") " pod="openshift-monitoring/metrics-server-64f677f8cd-7x5kz"
Jul 17 04:51:21.665857 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.665792    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4ffh\" (UniqueName: \"kubernetes.io/projected/7ff86f69-6040-4003-89d8-84ed6d165084-kube-api-access-g4ffh\") pod \"metrics-server-64f677f8cd-7x5kz\" (UID: \"7ff86f69-6040-4003-89d8-84ed6d165084\") " pod="openshift-monitoring/metrics-server-64f677f8cd-7x5kz"
Jul 17 04:51:21.775714 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.775601    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-64f677f8cd-7x5kz"
Jul 17 04:51:21.877832 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.877799    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-678d577485-gl4nr"]
Jul 17 04:51:21.883049 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.882709    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-678d577485-gl4nr"
Jul 17 04:51:21.886292 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.884924    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kzwxs"]
Jul 17 04:51:21.888930 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.888902    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kzwxs"
Jul 17 04:51:21.898233 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.897789    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-678d577485-gl4nr"]
Jul 17 04:51:21.898233 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.898035    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 17 04:51:21.898408 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.898300    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-svnx8\""
Jul 17 04:51:21.900124 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.900073    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kzwxs"]
Jul 17 04:51:21.955667 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.955266    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4f2f4ecb-c534-424e-8852-523892470327-console-oauth-config\") pod \"console-678d577485-gl4nr\" (UID: \"4f2f4ecb-c534-424e-8852-523892470327\") " pod="openshift-console/console-678d577485-gl4nr"
Jul 17 04:51:21.955667 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.955310    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4f2f4ecb-c534-424e-8852-523892470327-console-config\") pod \"console-678d577485-gl4nr\" (UID: \"4f2f4ecb-c534-424e-8852-523892470327\") " pod="openshift-console/console-678d577485-gl4nr"
Jul 17 04:51:21.955667 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.955392    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f2f4ecb-c534-424e-8852-523892470327-trusted-ca-bundle\") pod \"console-678d577485-gl4nr\" (UID: \"4f2f4ecb-c534-424e-8852-523892470327\") " pod="openshift-console/console-678d577485-gl4nr"
Jul 17 04:51:21.955667 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.955434    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f2f4ecb-c534-424e-8852-523892470327-console-serving-cert\") pod \"console-678d577485-gl4nr\" (UID: \"4f2f4ecb-c534-424e-8852-523892470327\") " pod="openshift-console/console-678d577485-gl4nr"
Jul 17 04:51:21.955667 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.955480    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4f2f4ecb-c534-424e-8852-523892470327-service-ca\") pod \"console-678d577485-gl4nr\" (UID: \"4f2f4ecb-c534-424e-8852-523892470327\") " pod="openshift-console/console-678d577485-gl4nr"
Jul 17 04:51:21.955667 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.955527    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5w78\" (UniqueName: \"kubernetes.io/projected/8c609620-1794-4451-8eaa-f9bc3440ea0b-kube-api-access-d5w78\") pod \"monitoring-plugin-7b7b7bc8bf-kzwxs\" (UID: \"8c609620-1794-4451-8eaa-f9bc3440ea0b\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kzwxs"
Jul 17 04:51:21.955667 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.955563    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4f2f4ecb-c534-424e-8852-523892470327-oauth-serving-cert\") pod \"console-678d577485-gl4nr\" (UID: \"4f2f4ecb-c534-424e-8852-523892470327\") " pod="openshift-console/console-678d577485-gl4nr"
Jul 17 04:51:21.956147 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.955588    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8c609620-1794-4451-8eaa-f9bc3440ea0b-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-kzwxs\" (UID: \"8c609620-1794-4451-8eaa-f9bc3440ea0b\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kzwxs"
Jul 17 04:51:21.956147 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.955736    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvnfv\" (UniqueName: \"kubernetes.io/projected/4f2f4ecb-c534-424e-8852-523892470327-kube-api-access-cvnfv\") pod \"console-678d577485-gl4nr\" (UID: \"4f2f4ecb-c534-424e-8852-523892470327\") " pod="openshift-console/console-678d577485-gl4nr"
Jul 17 04:51:21.987765 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:21.987689    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-64f677f8cd-7x5kz"]
Jul 17 04:51:21.997267 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:51:21.997230    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ff86f69_6040_4003_89d8_84ed6d165084.slice/crio-658c88a9fc1ecb8276569e531d888781b032244f662c764f937cd9ee8ff0f808 WatchSource:0}: Error finding container 658c88a9fc1ecb8276569e531d888781b032244f662c764f937cd9ee8ff0f808: Status 404 returned error can't find the container with id 658c88a9fc1ecb8276569e531d888781b032244f662c764f937cd9ee8ff0f808
Jul 17 04:51:22.056720 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:22.056681    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f2f4ecb-c534-424e-8852-523892470327-console-serving-cert\") pod \"console-678d577485-gl4nr\" (UID: \"4f2f4ecb-c534-424e-8852-523892470327\") " pod="openshift-console/console-678d577485-gl4nr"
Jul 17 04:51:22.056912 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:22.056756    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4f2f4ecb-c534-424e-8852-523892470327-service-ca\") pod \"console-678d577485-gl4nr\" (UID: \"4f2f4ecb-c534-424e-8852-523892470327\") " pod="openshift-console/console-678d577485-gl4nr"
Jul 17 04:51:22.056912 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:22.056809    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5w78\" (UniqueName: \"kubernetes.io/projected/8c609620-1794-4451-8eaa-f9bc3440ea0b-kube-api-access-d5w78\") pod \"monitoring-plugin-7b7b7bc8bf-kzwxs\" (UID: \"8c609620-1794-4451-8eaa-f9bc3440ea0b\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kzwxs"
Jul 17 04:51:22.057072 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:22.056987    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4f2f4ecb-c534-424e-8852-523892470327-oauth-serving-cert\") pod \"console-678d577485-gl4nr\" (UID: \"4f2f4ecb-c534-424e-8852-523892470327\") " pod="openshift-console/console-678d577485-gl4nr"
Jul 17 04:51:22.057072 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:22.057043    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8c609620-1794-4451-8eaa-f9bc3440ea0b-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-kzwxs\" (UID: \"8c609620-1794-4451-8eaa-f9bc3440ea0b\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kzwxs"
Jul 17 04:51:22.057182 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:22.057075    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvnfv\" (UniqueName: \"kubernetes.io/projected/4f2f4ecb-c534-424e-8852-523892470327-kube-api-access-cvnfv\") pod \"console-678d577485-gl4nr\" (UID: \"4f2f4ecb-c534-424e-8852-523892470327\") " pod="openshift-console/console-678d577485-gl4nr"
Jul 17 04:51:22.057182 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:22.057124    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4f2f4ecb-c534-424e-8852-523892470327-console-oauth-config\") pod \"console-678d577485-gl4nr\" (UID: \"4f2f4ecb-c534-424e-8852-523892470327\") " pod="openshift-console/console-678d577485-gl4nr"
Jul 17 04:51:22.057182 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:22.057150    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4f2f4ecb-c534-424e-8852-523892470327-console-config\") pod \"console-678d577485-gl4nr\" (UID: \"4f2f4ecb-c534-424e-8852-523892470327\") " pod="openshift-console/console-678d577485-gl4nr"
Jul 17 04:51:22.057343 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:22.057201    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f2f4ecb-c534-424e-8852-523892470327-trusted-ca-bundle\") pod \"console-678d577485-gl4nr\" (UID: \"4f2f4ecb-c534-424e-8852-523892470327\") " pod="openshift-console/console-678d577485-gl4nr"
Jul 17 04:51:22.057704 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:22.057653    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4f2f4ecb-c534-424e-8852-523892470327-service-ca\") pod \"console-678d577485-gl4nr\" (UID: \"4f2f4ecb-c534-424e-8852-523892470327\") " pod="openshift-console/console-678d577485-gl4nr"
Jul 17 04:51:22.058164 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:22.058062    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f2f4ecb-c534-424e-8852-523892470327-trusted-ca-bundle\") pod \"console-678d577485-gl4nr\" (UID: \"4f2f4ecb-c534-424e-8852-523892470327\") " pod="openshift-console/console-678d577485-gl4nr"
Jul 17 04:51:22.058520 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:22.058499    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4f2f4ecb-c534-424e-8852-523892470327-console-config\") pod \"console-678d577485-gl4nr\" (UID: \"4f2f4ecb-c534-424e-8852-523892470327\") " pod="openshift-console/console-678d577485-gl4nr"
Jul 17 04:51:22.058904 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:22.058879    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4f2f4ecb-c534-424e-8852-523892470327-oauth-serving-cert\") pod \"console-678d577485-gl4nr\" (UID: \"4f2f4ecb-c534-424e-8852-523892470327\") " pod="openshift-console/console-678d577485-gl4nr"
Jul 17 04:51:22.060139 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:22.060116    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f2f4ecb-c534-424e-8852-523892470327-console-serving-cert\") pod \"console-678d577485-gl4nr\" (UID: \"4f2f4ecb-c534-424e-8852-523892470327\") " pod="openshift-console/console-678d577485-gl4nr"
Jul 17 04:51:22.060511 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:22.060490    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4f2f4ecb-c534-424e-8852-523892470327-console-oauth-config\") pod \"console-678d577485-gl4nr\" (UID: \"4f2f4ecb-c534-424e-8852-523892470327\") " pod="openshift-console/console-678d577485-gl4nr"
Jul 17 04:51:22.060606 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:22.060587    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8c609620-1794-4451-8eaa-f9bc3440ea0b-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-kzwxs\" (UID: \"8c609620-1794-4451-8eaa-f9bc3440ea0b\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kzwxs"
Jul 17 04:51:22.065408 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:22.065365    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5w78\" (UniqueName: \"kubernetes.io/projected/8c609620-1794-4451-8eaa-f9bc3440ea0b-kube-api-access-d5w78\") pod \"monitoring-plugin-7b7b7bc8bf-kzwxs\" (UID: \"8c609620-1794-4451-8eaa-f9bc3440ea0b\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kzwxs"
Jul 17 04:51:22.066111 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:22.066086    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvnfv\" (UniqueName: \"kubernetes.io/projected/4f2f4ecb-c534-424e-8852-523892470327-kube-api-access-cvnfv\") pod \"console-678d577485-gl4nr\" (UID: \"4f2f4ecb-c534-424e-8852-523892470327\") " pod="openshift-console/console-678d577485-gl4nr"
Jul 17 04:51:22.209397 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:22.209357    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-678d577485-gl4nr"
Jul 17 04:51:22.217105 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:22.217074    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kzwxs"
Jul 17 04:51:22.302869 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:22.302746    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-64f677f8cd-7x5kz" event={"ID":"7ff86f69-6040-4003-89d8-84ed6d165084","Type":"ContainerStarted","Data":"658c88a9fc1ecb8276569e531d888781b032244f662c764f937cd9ee8ff0f808"}
Jul 17 04:51:22.305844 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:22.305815    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jtngp" event={"ID":"a16c8d78-1778-44b1-acb1-f8e609e3b331","Type":"ContainerStarted","Data":"6e7cba1fafe01002093c5cba2bc7754db1f00fcfbdc706a1cd7b51f777cd0524"}
Jul 17 04:51:22.306074 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:22.305866    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jtngp" event={"ID":"a16c8d78-1778-44b1-acb1-f8e609e3b331","Type":"ContainerStarted","Data":"827285e5792a3140697ace9f44a064baf5ebb850caf182656af6a35b8aa8c1a2"}
Jul 17 04:51:22.308889 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:22.308864    2562 generic.go:356] "Generic (PLEG): container finished" podID="2975ade8-a4ab-4bec-9781-64655a501d21" containerID="530e321e16958db1f8f1bf8b48b06a60e95a2ec9346a491cf34d5cd53e4f790a" exitCode=0
Jul 17 04:51:22.310940 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:22.309190    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2975ade8-a4ab-4bec-9781-64655a501d21","Type":"ContainerDied","Data":"530e321e16958db1f8f1bf8b48b06a60e95a2ec9346a491cf34d5cd53e4f790a"}
Jul 17 04:51:22.327541 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:22.327455    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-jtngp" podStartSLOduration=3.568732679 podStartE2EDuration="5.327420836s" totalImagesPullingTime="1.624129107s" totalInitContainerRuntime="134.55905ms" isStatefulPod=true podCreationTimestamp="2026-07-17 04:51:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 04:51:22.326260597 +0000 UTC m=+70.009854281" watchObservedRunningTime="2026-07-17 04:51:22.327420836 +0000 UTC m=+70.011014495"
Jul 17 04:51:23.384739 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.384697    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 17 04:51:23.389623 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.389597    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.392552 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.392529    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 17 04:51:23.392858 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.392839    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 17 04:51:23.393204 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.393188    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 17 04:51:23.393809 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.393792    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 17 04:51:23.394105 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.393901    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 17 04:51:23.394559 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.394538    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-5fr46\""
Jul 17 04:51:23.394795 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.394738    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 17 04:51:23.394997 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.394981    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 17 04:51:23.395464 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.395386    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 17 04:51:23.396113 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.395388    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 17 04:51:23.396601 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.395474    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-ctvlklecrdf00\""
Jul 17 04:51:23.397223 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.397149    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 17 04:51:23.397642 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.397578    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 17 04:51:23.397789 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.397770    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 17 04:51:23.399696 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.399672    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 17 04:51:23.406851 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.406794    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 17 04:51:23.409670 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.409578    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 17 04:51:23.474153 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.474103    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b6151930-7f52-4dad-8aa4-553bcad70b77-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.474292 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.474170    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6151930-7f52-4dad-8aa4-553bcad70b77-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.474292 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.474205    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b6151930-7f52-4dad-8aa4-553bcad70b77-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.474292 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.474267    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b6151930-7f52-4dad-8aa4-553bcad70b77-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.474401 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.474294    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b6151930-7f52-4dad-8aa4-553bcad70b77-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.474401 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.474320    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b6151930-7f52-4dad-8aa4-553bcad70b77-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.474401 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.474346    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b6151930-7f52-4dad-8aa4-553bcad70b77-web-config\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.474401 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.474385    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b6151930-7f52-4dad-8aa4-553bcad70b77-config-out\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.474518 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.474413    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kgbs\" (UniqueName: \"kubernetes.io/projected/b6151930-7f52-4dad-8aa4-553bcad70b77-kube-api-access-7kgbs\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.474518 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.474440    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b6151930-7f52-4dad-8aa4-553bcad70b77-config\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.474518 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.474470    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b6151930-7f52-4dad-8aa4-553bcad70b77-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.474518 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.474494    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b6151930-7f52-4dad-8aa4-553bcad70b77-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.474635 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.474523    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b6151930-7f52-4dad-8aa4-553bcad70b77-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.474635 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.474568    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b6151930-7f52-4dad-8aa4-553bcad70b77-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.474635 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.474592    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b6151930-7f52-4dad-8aa4-553bcad70b77-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.474635 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.474624    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b6151930-7f52-4dad-8aa4-553bcad70b77-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.474787 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.474657    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b6151930-7f52-4dad-8aa4-553bcad70b77-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.474787 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.474685    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6151930-7f52-4dad-8aa4-553bcad70b77-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.474787 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.474715    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6151930-7f52-4dad-8aa4-553bcad70b77-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.474787 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.474760    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6151930-7f52-4dad-8aa4-553bcad70b77-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.575663 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.575613    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6151930-7f52-4dad-8aa4-553bcad70b77-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.575663 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.575667    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b6151930-7f52-4dad-8aa4-553bcad70b77-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.575883 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.575709    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b6151930-7f52-4dad-8aa4-553bcad70b77-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.575883 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.575729    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b6151930-7f52-4dad-8aa4-553bcad70b77-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.575883 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.575753    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b6151930-7f52-4dad-8aa4-553bcad70b77-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.575883 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.575777    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b6151930-7f52-4dad-8aa4-553bcad70b77-web-config\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.575883 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.575815    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b6151930-7f52-4dad-8aa4-553bcad70b77-config-out\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.575883 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.575849    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kgbs\" (UniqueName: \"kubernetes.io/projected/b6151930-7f52-4dad-8aa4-553bcad70b77-kube-api-access-7kgbs\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.575883 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.575879    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b6151930-7f52-4dad-8aa4-553bcad70b77-config\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.577236 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.575911    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b6151930-7f52-4dad-8aa4-553bcad70b77-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.577236 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.575935    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b6151930-7f52-4dad-8aa4-553bcad70b77-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.577236 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.575982    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b6151930-7f52-4dad-8aa4-553bcad70b77-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.577236 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.576026    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b6151930-7f52-4dad-8aa4-553bcad70b77-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.577236 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.576050    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b6151930-7f52-4dad-8aa4-553bcad70b77-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.577236 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.576099    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b6151930-7f52-4dad-8aa4-553bcad70b77-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.577236 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.576130    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b6151930-7f52-4dad-8aa4-553bcad70b77-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.577236 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.576155    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6151930-7f52-4dad-8aa4-553bcad70b77-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.577236 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.576181    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6151930-7f52-4dad-8aa4-553bcad70b77-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.577236 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.576229    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6151930-7f52-4dad-8aa4-553bcad70b77-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.577236 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.576259    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b6151930-7f52-4dad-8aa4-553bcad70b77-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.577236 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.576777    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6151930-7f52-4dad-8aa4-553bcad70b77-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.577756 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.577275    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6151930-7f52-4dad-8aa4-553bcad70b77-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.583487 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.583078    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6151930-7f52-4dad-8aa4-553bcad70b77-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.583753 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.583728    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b6151930-7f52-4dad-8aa4-553bcad70b77-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.583922 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.583903    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b6151930-7f52-4dad-8aa4-553bcad70b77-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.585462 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.585160    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6151930-7f52-4dad-8aa4-553bcad70b77-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.585462 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.585424    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b6151930-7f52-4dad-8aa4-553bcad70b77-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.588867 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.588836    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b6151930-7f52-4dad-8aa4-553bcad70b77-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.589506 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.589160    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b6151930-7f52-4dad-8aa4-553bcad70b77-config-out\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.589506 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.589317    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b6151930-7f52-4dad-8aa4-553bcad70b77-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.589506 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.589417    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b6151930-7f52-4dad-8aa4-553bcad70b77-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.590052 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.589856    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b6151930-7f52-4dad-8aa4-553bcad70b77-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.590052 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.590005    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b6151930-7f52-4dad-8aa4-553bcad70b77-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.590208 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.590184    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b6151930-7f52-4dad-8aa4-553bcad70b77-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.591456 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.590696    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b6151930-7f52-4dad-8aa4-553bcad70b77-web-config\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.591456 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.590946    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/b6151930-7f52-4dad-8aa4-553bcad70b77-config\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.591456 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.591206    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b6151930-7f52-4dad-8aa4-553bcad70b77-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.594762 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.591530    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b6151930-7f52-4dad-8aa4-553bcad70b77-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.595342 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.595319    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b6151930-7f52-4dad-8aa4-553bcad70b77-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.596401 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.596379    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kgbs\" (UniqueName: \"kubernetes.io/projected/b6151930-7f52-4dad-8aa4-553bcad70b77-kube-api-access-7kgbs\") pod \"prometheus-k8s-0\" (UID: \"b6151930-7f52-4dad-8aa4-553bcad70b77\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:23.725099 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:23.724994    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:25.904826 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:25.904795    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kzwxs"]
Jul 17 04:51:26.136696 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:26.136617    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-678d577485-gl4nr"]
Jul 17 04:51:26.143493 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:26.143471    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 17 04:51:26.204463 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:51:26.204418    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6151930_7f52_4dad_8aa4_553bcad70b77.slice/crio-2016eb7c8641350103a6edb2bb7c644d59767b740e35f4d9a4381d63f0eb3be3 WatchSource:0}: Error finding container 2016eb7c8641350103a6edb2bb7c644d59767b740e35f4d9a4381d63f0eb3be3: Status 404 returned error can't find the container with id 2016eb7c8641350103a6edb2bb7c644d59767b740e35f4d9a4381d63f0eb3be3
Jul 17 04:51:26.204865 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:51:26.204828    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f2f4ecb_c534_424e_8852_523892470327.slice/crio-9c78400cfff6fad789c1452670744a6fb223c95f1eca499669e78d678541ead3 WatchSource:0}: Error finding container 9c78400cfff6fad789c1452670744a6fb223c95f1eca499669e78d678541ead3: Status 404 returned error can't find the container with id 9c78400cfff6fad789c1452670744a6fb223c95f1eca499669e78d678541ead3
Jul 17 04:51:26.206157 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:51:26.206134    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c609620_1794_4451_8eaa_f9bc3440ea0b.slice/crio-6112b85cafad7b3d713e7c0b461c70debd9447e1eca4e8e5b28a007112cefa70 WatchSource:0}: Error finding container 6112b85cafad7b3d713e7c0b461c70debd9447e1eca4e8e5b28a007112cefa70: Status 404 returned error can't find the container with id 6112b85cafad7b3d713e7c0b461c70debd9447e1eca4e8e5b28a007112cefa70
Jul 17 04:51:26.328411 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:26.328371    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kzwxs" event={"ID":"8c609620-1794-4451-8eaa-f9bc3440ea0b","Type":"ContainerStarted","Data":"6112b85cafad7b3d713e7c0b461c70debd9447e1eca4e8e5b28a007112cefa70"}
Jul 17 04:51:26.329888 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:26.329847    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-678d577485-gl4nr" event={"ID":"4f2f4ecb-c534-424e-8852-523892470327","Type":"ContainerStarted","Data":"9c78400cfff6fad789c1452670744a6fb223c95f1eca499669e78d678541ead3"}
Jul 17 04:51:26.331233 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:26.331209    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b6151930-7f52-4dad-8aa4-553bcad70b77","Type":"ContainerStarted","Data":"2016eb7c8641350103a6edb2bb7c644d59767b740e35f4d9a4381d63f0eb3be3"}
Jul 17 04:51:27.337399 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:27.337365    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7875fbb676-p8ln5" event={"ID":"679b67c5-405f-4561-9846-8aaef06ee7bc","Type":"ContainerStarted","Data":"a28ef2d6b9b057b5af5f0f20ce6253fc29e6e6bdb2593823b3b465f7d73d4995"}
Jul 17 04:51:27.337785 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:27.337441    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7875fbb676-p8ln5" event={"ID":"679b67c5-405f-4561-9846-8aaef06ee7bc","Type":"ContainerStarted","Data":"415670c597b224d2dff1a86e2ab2a2d0285a099081510cc3db740d881ebab509"}
Jul 17 04:51:27.337785 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:27.337469    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7875fbb676-p8ln5" event={"ID":"679b67c5-405f-4561-9846-8aaef06ee7bc","Type":"ContainerStarted","Data":"9585338766a1d3f3ebe7f64faefdcd405c60bb10632362d7af94434ebfc1cee3"}
Jul 17 04:51:27.340548 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:27.340491    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2975ade8-a4ab-4bec-9781-64655a501d21","Type":"ContainerStarted","Data":"a07f61dd1b0ed3e8f68bfff763b62f6c4e37ff90e081afb892af6bcc82f1cf32"}
Jul 17 04:51:27.340647 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:27.340600    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2975ade8-a4ab-4bec-9781-64655a501d21","Type":"ContainerStarted","Data":"503ff1be9282ca9695095b90f127f72150d76712446a54013fc6f3454b3fade8"}
Jul 17 04:51:27.340647 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:27.340630    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2975ade8-a4ab-4bec-9781-64655a501d21","Type":"ContainerStarted","Data":"8f0318501844ad955d75a159ff323f425a0165b545d02fa2f63a6dd690cdfd0d"}
Jul 17 04:51:27.340745 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:27.340655    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2975ade8-a4ab-4bec-9781-64655a501d21","Type":"ContainerStarted","Data":"9546ae6cc066c4326c8eee4af4cbe51bbbfd4d4a7598794fea3f62d20696561d"}
Jul 17 04:51:27.340745 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:27.340684    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2975ade8-a4ab-4bec-9781-64655a501d21","Type":"ContainerStarted","Data":"ca24fb2cbeddb23dedd56831face4d62479f35e8aecf6a05f8d75c70f25ccd3e"}
Jul 17 04:51:27.341887 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:27.341862    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6bkt5" event={"ID":"237abccc-c19c-4fca-be94-d3013f3c2e86","Type":"ContainerStarted","Data":"e6bfe7cbc0b13ec728e195453c08c98b5370972f26361276be4044d61981560f"}
Jul 17 04:51:27.343244 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:27.343221    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9vmqd" event={"ID":"bcee811d-1474-45f6-b0c7-afceee510ccd","Type":"ContainerStarted","Data":"0e1ee449ef40df13de203c297fb6357c5a326916c647f73dfa74f42a9d8a2b14"}
Jul 17 04:51:27.343348 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:27.343272    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-9vmqd"
Jul 17 04:51:27.344632 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:27.344610    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-64f677f8cd-7x5kz" event={"ID":"7ff86f69-6040-4003-89d8-84ed6d165084","Type":"ContainerStarted","Data":"2861bb89ed6e9d3fc16ae441b8d5c188b49d3f9f05a58fa097d29ef2e97bb693"}
Jul 17 04:51:27.346102 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:27.346071    2562 generic.go:356] "Generic (PLEG): container finished" podID="b6151930-7f52-4dad-8aa4-553bcad70b77" containerID="3a83928c56e246ba52b18617b55e29f78b526dc6af285d8f566c58d0628c80af" exitCode=0
Jul 17 04:51:27.346180 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:27.346144    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b6151930-7f52-4dad-8aa4-553bcad70b77","Type":"ContainerDied","Data":"3a83928c56e246ba52b18617b55e29f78b526dc6af285d8f566c58d0628c80af"}
Jul 17 04:51:27.350133 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:27.350022    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-678d577485-gl4nr" event={"ID":"4f2f4ecb-c534-424e-8852-523892470327","Type":"ContainerStarted","Data":"40274d1ac911f360669f4e8bf7e31e9978171338fc0b2e4b079f64f1d4a28f03"}
Jul 17 04:51:27.357421 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:27.357385    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-6bkt5" podStartSLOduration=66.851193915 podStartE2EDuration="1m14.357373495s" totalImagesPullingTime="7.50617958s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 04:50:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 04:51:27.356030486 +0000 UTC m=+75.039624166" watchObservedRunningTime="2026-07-17 04:51:27.357373495 +0000 UTC m=+75.040967130"
Jul 17 04:51:27.421276 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:27.421231    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-678d577485-gl4nr" podStartSLOduration=6.421215136 podStartE2EDuration="6.421215136s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 04:51:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 04:51:27.419767544 +0000 UTC m=+75.103361226" watchObservedRunningTime="2026-07-17 04:51:27.421215136 +0000 UTC m=+75.104808798"
Jul 17 04:51:27.421862 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:27.421824    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-9vmqd" podStartSLOduration=68.783475264 podStartE2EDuration="1m15.421807784s" totalImagesPullingTime="6.63833252s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 04:50:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 04:51:27.400386787 +0000 UTC m=+75.083980463" watchObservedRunningTime="2026-07-17 04:51:27.421807784 +0000 UTC m=+75.105401437"
Jul 17 04:51:27.442772 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:27.442721    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-64f677f8cd-7x5kz" podStartSLOduration=1.852393741 podStartE2EDuration="6.442703866s" totalImagesPullingTime="4.590310125s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 04:51:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 04:51:27.441011117 +0000 UTC m=+75.124604794" watchObservedRunningTime="2026-07-17 04:51:27.442703866 +0000 UTC m=+75.126297520"
Jul 17 04:51:28.359715 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:28.359660    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7875fbb676-p8ln5" event={"ID":"679b67c5-405f-4561-9846-8aaef06ee7bc","Type":"ContainerStarted","Data":"818a03b71f01877537035c811e89752ae8dbba59b0313ae1e95e40d440c73cbb"}
Jul 17 04:51:28.360105 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:28.359746    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7875fbb676-p8ln5" event={"ID":"679b67c5-405f-4561-9846-8aaef06ee7bc","Type":"ContainerStarted","Data":"93c49c70cf3e07eb76c34fe789fd06b2de57a305bf1e9f103927c5c356ebc626"}
Jul 17 04:51:28.360105 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:28.359790    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7875fbb676-p8ln5" event={"ID":"679b67c5-405f-4561-9846-8aaef06ee7bc","Type":"ContainerStarted","Data":"8fb618bbcb2cb065bbc7aca0a0a8f10d78ee869f0de5faee38d4b4fda3565259"}
Jul 17 04:51:28.360105 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:28.359917    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7875fbb676-p8ln5"
Jul 17 04:51:28.365124 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:28.365067    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2975ade8-a4ab-4bec-9781-64655a501d21","Type":"ContainerStarted","Data":"fbb4cbe44ae0984b007b4fbe379b5d1d654e319583c5990674213d625f1f7861"}
Jul 17 04:51:28.367809 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:28.367784    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kzwxs" event={"ID":"8c609620-1794-4451-8eaa-f9bc3440ea0b","Type":"ContainerStarted","Data":"150aa8ffa1e0846083126397dc9ced893580bbebfa0bcd4468ed7a7c58873639"}
Jul 17 04:51:28.385510 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:28.385466    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7875fbb676-p8ln5" podStartSLOduration=2.504610647 podStartE2EDuration="9.385448128s" totalImagesPullingTime="6.880837481s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 04:51:19 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 04:51:28.380500186 +0000 UTC m=+76.064093866" watchObservedRunningTime="2026-07-17 04:51:28.385448128 +0000 UTC m=+76.069041787"
Jul 17 04:51:28.396499 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:28.396456    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kzwxs" podStartSLOduration=5.525760819 podStartE2EDuration="7.396443563s" totalImagesPullingTime="1.870682744s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 04:51:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 04:51:28.396219421 +0000 UTC m=+76.079813105" watchObservedRunningTime="2026-07-17 04:51:28.396443563 +0000 UTC m=+76.080037235"
Jul 17 04:51:28.421538 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:28.421492    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.602011739 podStartE2EDuration="10.421481412s" totalImagesPullingTime="7.789562783s" totalInitContainerRuntime="29.90689ms" isStatefulPod=false podCreationTimestamp="2026-07-17 04:51:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 04:51:28.419881571 +0000 UTC m=+76.103475256" watchObservedRunningTime="2026-07-17 04:51:28.421481412 +0000 UTC m=+76.105075069"
Jul 17 04:51:29.372801 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:29.372767    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kzwxs"
Jul 17 04:51:29.378161 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:29.378138    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-kzwxs"
Jul 17 04:51:31.385495 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:31.385456    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b6151930-7f52-4dad-8aa4-553bcad70b77","Type":"ContainerStarted","Data":"a0916a3f02f30779af5e748649fe3a2c7aa55003785335ae49e7760178a6ef37"}
Jul 17 04:51:31.385888 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:31.385538    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b6151930-7f52-4dad-8aa4-553bcad70b77","Type":"ContainerStarted","Data":"825f19942d166a59e742deca9c690c3c228fe0c6a3d2055f5c67d8e75af7e132"}
Jul 17 04:51:31.385888 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:31.385578    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b6151930-7f52-4dad-8aa4-553bcad70b77","Type":"ContainerStarted","Data":"1821296126f864225f323b89c8c2a296ed71c7b510c42b252e57efaa519e8cab"}
Jul 17 04:51:31.385888 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:31.385620    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b6151930-7f52-4dad-8aa4-553bcad70b77","Type":"ContainerStarted","Data":"d40b2cfddffa97c6d13c19a1db3e269d08e06759a43b42071b68b3435d7bc894"}
Jul 17 04:51:31.385888 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:31.385663    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b6151930-7f52-4dad-8aa4-553bcad70b77","Type":"ContainerStarted","Data":"e666dca6086767f93dc90aeb8b776e6269a74afd1d111ab6dd9b4add1bdea134"}
Jul 17 04:51:31.385888 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:31.385697    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b6151930-7f52-4dad-8aa4-553bcad70b77","Type":"ContainerStarted","Data":"21fbaf3c6f41e8e8ec902ea244ec9f26972ede476503fb7a2363cb53151dbd5b"}
Jul 17 04:51:31.418953 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:31.418906    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=5.091757275 podStartE2EDuration="8.41888698s" totalImagesPullingTime="3.277435414s" totalInitContainerRuntime="49.694291ms" isStatefulPod=false podCreationTimestamp="2026-07-17 04:51:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 04:51:31.416569906 +0000 UTC m=+79.100163585" watchObservedRunningTime="2026-07-17 04:51:31.41888698 +0000 UTC m=+79.102480829"
Jul 17 04:51:32.209980 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:32.209937    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-678d577485-gl4nr"
Jul 17 04:51:32.210195 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:32.210029    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-678d577485-gl4nr"
Jul 17 04:51:32.214815 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:32.214794    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-678d577485-gl4nr"
Jul 17 04:51:32.393757 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:32.393727    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-678d577485-gl4nr"
Jul 17 04:51:32.437956 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:32.437930    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75ff58c455-pksb8"]
Jul 17 04:51:33.726037 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:33.725916    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:51:34.379342 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:34.379317    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7875fbb676-p8ln5"
Jul 17 04:51:41.775917 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:41.775879    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-64f677f8cd-7x5kz"
Jul 17 04:51:41.776384 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:41.775941    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-64f677f8cd-7x5kz"
Jul 17 04:51:57.458931 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:57.458884    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-75ff58c455-pksb8" podUID="5a143a72-1103-45f5-97b1-18bcb51c06bd" containerName="console" containerID="cri-o://924c1a8a1e0595154ed9677a1d93d8b62fc78391d4b327b7b6d4daabf22e35bb" gracePeriod=15
Jul 17 04:51:57.692766 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:57.692743    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75ff58c455-pksb8"
Jul 17 04:51:57.786630 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:57.786559    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a143a72-1103-45f5-97b1-18bcb51c06bd-oauth-serving-cert\") pod \"5a143a72-1103-45f5-97b1-18bcb51c06bd\" (UID: \"5a143a72-1103-45f5-97b1-18bcb51c06bd\") "
Jul 17 04:51:57.786630 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:57.786602    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a143a72-1103-45f5-97b1-18bcb51c06bd-console-config\") pod \"5a143a72-1103-45f5-97b1-18bcb51c06bd\" (UID: \"5a143a72-1103-45f5-97b1-18bcb51c06bd\") "
Jul 17 04:51:57.786630 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:57.786627    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-njq58\" (UniqueName: \"kubernetes.io/projected/5a143a72-1103-45f5-97b1-18bcb51c06bd-kube-api-access-njq58\") pod \"5a143a72-1103-45f5-97b1-18bcb51c06bd\" (UID: \"5a143a72-1103-45f5-97b1-18bcb51c06bd\") "
Jul 17 04:51:57.786882 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:57.786647    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a143a72-1103-45f5-97b1-18bcb51c06bd-console-serving-cert\") pod \"5a143a72-1103-45f5-97b1-18bcb51c06bd\" (UID: \"5a143a72-1103-45f5-97b1-18bcb51c06bd\") "
Jul 17 04:51:57.786882 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:57.786694    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a143a72-1103-45f5-97b1-18bcb51c06bd-console-oauth-config\") pod \"5a143a72-1103-45f5-97b1-18bcb51c06bd\" (UID: \"5a143a72-1103-45f5-97b1-18bcb51c06bd\") "
Jul 17 04:51:57.786882 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:57.786709    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a143a72-1103-45f5-97b1-18bcb51c06bd-trusted-ca-bundle\") pod \"5a143a72-1103-45f5-97b1-18bcb51c06bd\" (UID: \"5a143a72-1103-45f5-97b1-18bcb51c06bd\") "
Jul 17 04:51:57.786882 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:57.786740    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a143a72-1103-45f5-97b1-18bcb51c06bd-service-ca\") pod \"5a143a72-1103-45f5-97b1-18bcb51c06bd\" (UID: \"5a143a72-1103-45f5-97b1-18bcb51c06bd\") "
Jul 17 04:51:57.787129 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:57.787022    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a143a72-1103-45f5-97b1-18bcb51c06bd-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5a143a72-1103-45f5-97b1-18bcb51c06bd" (UID: "5a143a72-1103-45f5-97b1-18bcb51c06bd"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 04:51:57.787129 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:57.787018    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a143a72-1103-45f5-97b1-18bcb51c06bd-console-config" (OuterVolumeSpecName: "console-config") pod "5a143a72-1103-45f5-97b1-18bcb51c06bd" (UID: "5a143a72-1103-45f5-97b1-18bcb51c06bd"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 04:51:57.787324 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:57.787301    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a143a72-1103-45f5-97b1-18bcb51c06bd-service-ca" (OuterVolumeSpecName: "service-ca") pod "5a143a72-1103-45f5-97b1-18bcb51c06bd" (UID: "5a143a72-1103-45f5-97b1-18bcb51c06bd"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 04:51:57.787551 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:57.787518    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a143a72-1103-45f5-97b1-18bcb51c06bd-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5a143a72-1103-45f5-97b1-18bcb51c06bd" (UID: "5a143a72-1103-45f5-97b1-18bcb51c06bd"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 04:51:57.789240 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:57.789210    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a143a72-1103-45f5-97b1-18bcb51c06bd-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5a143a72-1103-45f5-97b1-18bcb51c06bd" (UID: "5a143a72-1103-45f5-97b1-18bcb51c06bd"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 04:51:57.789240 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:57.789224    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a143a72-1103-45f5-97b1-18bcb51c06bd-kube-api-access-njq58" (OuterVolumeSpecName: "kube-api-access-njq58") pod "5a143a72-1103-45f5-97b1-18bcb51c06bd" (UID: "5a143a72-1103-45f5-97b1-18bcb51c06bd"). InnerVolumeSpecName "kube-api-access-njq58". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 04:51:57.789372 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:57.789285    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a143a72-1103-45f5-97b1-18bcb51c06bd-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5a143a72-1103-45f5-97b1-18bcb51c06bd" (UID: "5a143a72-1103-45f5-97b1-18bcb51c06bd"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 04:51:57.887620 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:57.887572    2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a143a72-1103-45f5-97b1-18bcb51c06bd-console-config\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 04:51:57.887620 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:57.887615    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njq58\" (UniqueName: \"kubernetes.io/projected/5a143a72-1103-45f5-97b1-18bcb51c06bd-kube-api-access-njq58\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 04:51:57.887620 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:57.887629    2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a143a72-1103-45f5-97b1-18bcb51c06bd-console-serving-cert\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 04:51:57.887843 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:57.887640    2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a143a72-1103-45f5-97b1-18bcb51c06bd-console-oauth-config\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 04:51:57.887843 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:57.887652    2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a143a72-1103-45f5-97b1-18bcb51c06bd-trusted-ca-bundle\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 04:51:57.887843 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:57.887662    2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a143a72-1103-45f5-97b1-18bcb51c06bd-service-ca\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 04:51:57.887843 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:57.887673    2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a143a72-1103-45f5-97b1-18bcb51c06bd-oauth-serving-cert\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 04:51:58.370598 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:58.370572    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-9vmqd"
Jul 17 04:51:58.496459 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:58.496417    2562 generic.go:356] "Generic (PLEG): container finished" podID="5a143a72-1103-45f5-97b1-18bcb51c06bd" containerID="924c1a8a1e0595154ed9677a1d93d8b62fc78391d4b327b7b6d4daabf22e35bb" exitCode=0
Jul 17 04:51:58.496459 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:58.496464    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75ff58c455-pksb8" event={"ID":"5a143a72-1103-45f5-97b1-18bcb51c06bd","Type":"ContainerDied","Data":"924c1a8a1e0595154ed9677a1d93d8b62fc78391d4b327b7b6d4daabf22e35bb"}
Jul 17 04:51:58.496991 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:58.496474    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75ff58c455-pksb8"
Jul 17 04:51:58.496991 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:58.496516    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75ff58c455-pksb8" event={"ID":"5a143a72-1103-45f5-97b1-18bcb51c06bd","Type":"ContainerDied","Data":"2fc47a04a80baac9088ca91e8132d1214796f0524b1ca2b34761b39f3e8ccd5a"}
Jul 17 04:51:58.496991 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:58.496547    2562 scope.go:117] "RemoveContainer" containerID="924c1a8a1e0595154ed9677a1d93d8b62fc78391d4b327b7b6d4daabf22e35bb"
Jul 17 04:51:58.505639 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:58.505621    2562 scope.go:117] "RemoveContainer" containerID="924c1a8a1e0595154ed9677a1d93d8b62fc78391d4b327b7b6d4daabf22e35bb"
Jul 17 04:51:58.505919 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:51:58.505901    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"924c1a8a1e0595154ed9677a1d93d8b62fc78391d4b327b7b6d4daabf22e35bb\": container with ID starting with 924c1a8a1e0595154ed9677a1d93d8b62fc78391d4b327b7b6d4daabf22e35bb not found: ID does not exist" containerID="924c1a8a1e0595154ed9677a1d93d8b62fc78391d4b327b7b6d4daabf22e35bb"
Jul 17 04:51:58.506033 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:58.505929    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"924c1a8a1e0595154ed9677a1d93d8b62fc78391d4b327b7b6d4daabf22e35bb"} err="failed to get container status \"924c1a8a1e0595154ed9677a1d93d8b62fc78391d4b327b7b6d4daabf22e35bb\": rpc error: code = NotFound desc = could not find container \"924c1a8a1e0595154ed9677a1d93d8b62fc78391d4b327b7b6d4daabf22e35bb\": container with ID starting with 924c1a8a1e0595154ed9677a1d93d8b62fc78391d4b327b7b6d4daabf22e35bb not found: ID does not exist"
Jul 17 04:51:58.517231 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:58.517208    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75ff58c455-pksb8"]
Jul 17 04:51:58.520557 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:58.520538    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-75ff58c455-pksb8"]
Jul 17 04:51:58.883181 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:51:58.883151    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a143a72-1103-45f5-97b1-18bcb51c06bd" path="/var/lib/kubelet/pods/5a143a72-1103-45f5-97b1-18bcb51c06bd/volumes"
Jul 17 04:52:01.783377 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:52:01.783348    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-64f677f8cd-7x5kz"
Jul 17 04:52:01.791098 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:52:01.791077    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-64f677f8cd-7x5kz"
Jul 17 04:52:23.725502 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:52:23.725465    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:52:23.745369 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:52:23.745337    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:52:24.624160 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:52:24.624126    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 04:52:57.980041 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:52:57.980005    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-678d577485-gl4nr"]
Jul 17 04:53:23.006258 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:53:23.006192    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-678d577485-gl4nr" podUID="4f2f4ecb-c534-424e-8852-523892470327" containerName="console" containerID="cri-o://40274d1ac911f360669f4e8bf7e31e9978171338fc0b2e4b079f64f1d4a28f03" gracePeriod=15
Jul 17 04:53:23.245642 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:53:23.245618    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-678d577485-gl4nr"
Jul 17 04:53:23.320773 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:53:23.320743    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4f2f4ecb-c534-424e-8852-523892470327-service-ca\") pod \"4f2f4ecb-c534-424e-8852-523892470327\" (UID: \"4f2f4ecb-c534-424e-8852-523892470327\") "
Jul 17 04:53:23.320945 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:53:23.320802    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f2f4ecb-c534-424e-8852-523892470327-console-serving-cert\") pod \"4f2f4ecb-c534-424e-8852-523892470327\" (UID: \"4f2f4ecb-c534-424e-8852-523892470327\") "
Jul 17 04:53:23.320945 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:53:23.320847    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4f2f4ecb-c534-424e-8852-523892470327-console-config\") pod \"4f2f4ecb-c534-424e-8852-523892470327\" (UID: \"4f2f4ecb-c534-424e-8852-523892470327\") "
Jul 17 04:53:23.320945 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:53:23.320886    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f2f4ecb-c534-424e-8852-523892470327-trusted-ca-bundle\") pod \"4f2f4ecb-c534-424e-8852-523892470327\" (UID: \"4f2f4ecb-c534-424e-8852-523892470327\") "
Jul 17 04:53:23.320945 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:53:23.320929    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4f2f4ecb-c534-424e-8852-523892470327-console-oauth-config\") pod \"4f2f4ecb-c534-424e-8852-523892470327\" (UID: \"4f2f4ecb-c534-424e-8852-523892470327\") "
Jul 17 04:53:23.321184 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:53:23.320983    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4f2f4ecb-c534-424e-8852-523892470327-oauth-serving-cert\") pod \"4f2f4ecb-c534-424e-8852-523892470327\" (UID: \"4f2f4ecb-c534-424e-8852-523892470327\") "
Jul 17 04:53:23.321338 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:53:23.321306    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f2f4ecb-c534-424e-8852-523892470327-console-config" (OuterVolumeSpecName: "console-config") pod "4f2f4ecb-c534-424e-8852-523892470327" (UID: "4f2f4ecb-c534-424e-8852-523892470327"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 04:53:23.321470 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:53:23.321434    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f2f4ecb-c534-424e-8852-523892470327-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4f2f4ecb-c534-424e-8852-523892470327" (UID: "4f2f4ecb-c534-424e-8852-523892470327"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 04:53:23.321470 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:53:23.321449    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f2f4ecb-c534-424e-8852-523892470327-service-ca" (OuterVolumeSpecName: "service-ca") pod "4f2f4ecb-c534-424e-8852-523892470327" (UID: "4f2f4ecb-c534-424e-8852-523892470327"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 04:53:23.321470 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:53:23.321443    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f2f4ecb-c534-424e-8852-523892470327-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4f2f4ecb-c534-424e-8852-523892470327" (UID: "4f2f4ecb-c534-424e-8852-523892470327"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 04:53:23.323303 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:53:23.323280    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f2f4ecb-c534-424e-8852-523892470327-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4f2f4ecb-c534-424e-8852-523892470327" (UID: "4f2f4ecb-c534-424e-8852-523892470327"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 04:53:23.323401 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:53:23.323344    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f2f4ecb-c534-424e-8852-523892470327-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4f2f4ecb-c534-424e-8852-523892470327" (UID: "4f2f4ecb-c534-424e-8852-523892470327"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 04:53:23.421494 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:53:23.421457    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cvnfv\" (UniqueName: \"kubernetes.io/projected/4f2f4ecb-c534-424e-8852-523892470327-kube-api-access-cvnfv\") pod \"4f2f4ecb-c534-424e-8852-523892470327\" (UID: \"4f2f4ecb-c534-424e-8852-523892470327\") "
Jul 17 04:53:23.421664 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:53:23.421626    2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f2f4ecb-c534-424e-8852-523892470327-console-serving-cert\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 04:53:23.421664 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:53:23.421637    2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4f2f4ecb-c534-424e-8852-523892470327-console-config\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 04:53:23.421664 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:53:23.421646    2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f2f4ecb-c534-424e-8852-523892470327-trusted-ca-bundle\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 04:53:23.421664 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:53:23.421656    2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4f2f4ecb-c534-424e-8852-523892470327-console-oauth-config\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 04:53:23.421664 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:53:23.421664    2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4f2f4ecb-c534-424e-8852-523892470327-oauth-serving-cert\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 04:53:23.421821 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:53:23.421675    2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4f2f4ecb-c534-424e-8852-523892470327-service-ca\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 04:53:23.423673 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:53:23.423649    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f2f4ecb-c534-424e-8852-523892470327-kube-api-access-cvnfv" (OuterVolumeSpecName: "kube-api-access-cvnfv") pod "4f2f4ecb-c534-424e-8852-523892470327" (UID: "4f2f4ecb-c534-424e-8852-523892470327"). InnerVolumeSpecName "kube-api-access-cvnfv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 04:53:23.522046 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:53:23.522017    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cvnfv\" (UniqueName: \"kubernetes.io/projected/4f2f4ecb-c534-424e-8852-523892470327-kube-api-access-cvnfv\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 04:53:23.845065 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:53:23.845028    2562 generic.go:356] "Generic (PLEG): container finished" podID="4f2f4ecb-c534-424e-8852-523892470327" containerID="40274d1ac911f360669f4e8bf7e31e9978171338fc0b2e4b079f64f1d4a28f03" exitCode=0
Jul 17 04:53:23.845240 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:53:23.845077    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-678d577485-gl4nr"
Jul 17 04:53:23.845240 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:53:23.845114    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-678d577485-gl4nr" event={"ID":"4f2f4ecb-c534-424e-8852-523892470327","Type":"ContainerDied","Data":"40274d1ac911f360669f4e8bf7e31e9978171338fc0b2e4b079f64f1d4a28f03"}
Jul 17 04:53:23.845240 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:53:23.845165    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-678d577485-gl4nr" event={"ID":"4f2f4ecb-c534-424e-8852-523892470327","Type":"ContainerDied","Data":"9c78400cfff6fad789c1452670744a6fb223c95f1eca499669e78d678541ead3"}
Jul 17 04:53:23.845240 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:53:23.845185    2562 scope.go:117] "RemoveContainer" containerID="40274d1ac911f360669f4e8bf7e31e9978171338fc0b2e4b079f64f1d4a28f03"
Jul 17 04:53:23.854051 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:53:23.854033    2562 scope.go:117] "RemoveContainer" containerID="40274d1ac911f360669f4e8bf7e31e9978171338fc0b2e4b079f64f1d4a28f03"
Jul 17 04:53:23.854335 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:53:23.854316    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40274d1ac911f360669f4e8bf7e31e9978171338fc0b2e4b079f64f1d4a28f03\": container with ID starting with 40274d1ac911f360669f4e8bf7e31e9978171338fc0b2e4b079f64f1d4a28f03 not found: ID does not exist" containerID="40274d1ac911f360669f4e8bf7e31e9978171338fc0b2e4b079f64f1d4a28f03"
Jul 17 04:53:23.854406 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:53:23.854345    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40274d1ac911f360669f4e8bf7e31e9978171338fc0b2e4b079f64f1d4a28f03"} err="failed to get container status \"40274d1ac911f360669f4e8bf7e31e9978171338fc0b2e4b079f64f1d4a28f03\": rpc error: code = NotFound desc = could not find container \"40274d1ac911f360669f4e8bf7e31e9978171338fc0b2e4b079f64f1d4a28f03\": container with ID starting with 40274d1ac911f360669f4e8bf7e31e9978171338fc0b2e4b079f64f1d4a28f03 not found: ID does not exist"
Jul 17 04:53:23.865285 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:53:23.865253    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-678d577485-gl4nr"]
Jul 17 04:53:23.866632 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:53:23.866611    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-678d577485-gl4nr"]
Jul 17 04:53:24.883433 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:53:24.883398    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f2f4ecb-c534-424e-8852-523892470327" path="/var/lib/kubelet/pods/4f2f4ecb-c534-424e-8852-523892470327/volumes"
Jul 17 04:54:13.413495 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:13.413456    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-ppmd8"]
Jul 17 04:54:13.414066 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:13.413917    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f2f4ecb-c534-424e-8852-523892470327" containerName="console"
Jul 17 04:54:13.414066 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:13.413936    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f2f4ecb-c534-424e-8852-523892470327" containerName="console"
Jul 17 04:54:13.414066 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:13.413988    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a143a72-1103-45f5-97b1-18bcb51c06bd" containerName="console"
Jul 17 04:54:13.414066 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:13.413998    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a143a72-1103-45f5-97b1-18bcb51c06bd" containerName="console"
Jul 17 04:54:13.414288 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:13.414070    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f2f4ecb-c534-424e-8852-523892470327" containerName="console"
Jul 17 04:54:13.414288 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:13.414088    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a143a72-1103-45f5-97b1-18bcb51c06bd" containerName="console"
Jul 17 04:54:13.418310 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:13.418288    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ppmd8"
Jul 17 04:54:13.420757 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:13.420737    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 17 04:54:13.420875 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:13.420740    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 17 04:54:13.421771 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:13.421748    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-2tj78\""
Jul 17 04:54:13.427928 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:13.427906    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-ppmd8"]
Jul 17 04:54:13.556646 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:13.556609    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vfck\" (UniqueName: \"kubernetes.io/projected/07021f4e-0824-4ffc-99bb-dd48834bd405-kube-api-access-4vfck\") pod \"cert-manager-cainjector-559cdd8c87-ppmd8\" (UID: \"07021f4e-0824-4ffc-99bb-dd48834bd405\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ppmd8"
Jul 17 04:54:13.556837 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:13.556661    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/07021f4e-0824-4ffc-99bb-dd48834bd405-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-ppmd8\" (UID: \"07021f4e-0824-4ffc-99bb-dd48834bd405\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ppmd8"
Jul 17 04:54:13.657977 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:13.657939    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vfck\" (UniqueName: \"kubernetes.io/projected/07021f4e-0824-4ffc-99bb-dd48834bd405-kube-api-access-4vfck\") pod \"cert-manager-cainjector-559cdd8c87-ppmd8\" (UID: \"07021f4e-0824-4ffc-99bb-dd48834bd405\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ppmd8"
Jul 17 04:54:13.658157 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:13.658045    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/07021f4e-0824-4ffc-99bb-dd48834bd405-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-ppmd8\" (UID: \"07021f4e-0824-4ffc-99bb-dd48834bd405\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ppmd8"
Jul 17 04:54:13.666197 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:13.666121    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/07021f4e-0824-4ffc-99bb-dd48834bd405-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-ppmd8\" (UID: \"07021f4e-0824-4ffc-99bb-dd48834bd405\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ppmd8"
Jul 17 04:54:13.666333 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:13.666309    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vfck\" (UniqueName: \"kubernetes.io/projected/07021f4e-0824-4ffc-99bb-dd48834bd405-kube-api-access-4vfck\") pod \"cert-manager-cainjector-559cdd8c87-ppmd8\" (UID: \"07021f4e-0824-4ffc-99bb-dd48834bd405\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ppmd8"
Jul 17 04:54:13.732573 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:13.732545    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ppmd8"
Jul 17 04:54:13.865215 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:13.865175    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-ppmd8"]
Jul 17 04:54:13.867990 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:54:13.867945    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07021f4e_0824_4ffc_99bb_dd48834bd405.slice/crio-0e39fb65ccddb99a9e7c42fbc5dd9963b2ecbbce44cbbcfc31f772dad5be7b3c WatchSource:0}: Error finding container 0e39fb65ccddb99a9e7c42fbc5dd9963b2ecbbce44cbbcfc31f772dad5be7b3c: Status 404 returned error can't find the container with id 0e39fb65ccddb99a9e7c42fbc5dd9963b2ecbbce44cbbcfc31f772dad5be7b3c
Jul 17 04:54:14.044654 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:14.044575    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ppmd8" event={"ID":"07021f4e-0824-4ffc-99bb-dd48834bd405","Type":"ContainerStarted","Data":"0e39fb65ccddb99a9e7c42fbc5dd9963b2ecbbce44cbbcfc31f772dad5be7b3c"}
Jul 17 04:54:17.060615 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:17.060573    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ppmd8" event={"ID":"07021f4e-0824-4ffc-99bb-dd48834bd405","Type":"ContainerStarted","Data":"f6103491bb1152a6ebd76536afee07402757ee8ec7836bd910344b6cbb6d3e75"}
Jul 17 04:54:17.079604 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:17.079472    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ppmd8" podStartSLOduration=1.067239701 podStartE2EDuration="4.079454521s" totalImagesPullingTime="3.01221482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 04:54:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 04:54:17.078890541 +0000 UTC m=+244.762484200" watchObservedRunningTime="2026-07-17 04:54:17.079454521 +0000 UTC m=+244.763048179"
Jul 17 04:54:21.397417 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:21.397376    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-bn76n"]
Jul 17 04:54:21.400766 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:21.400744    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-bn76n"
Jul 17 04:54:21.403057 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:21.403035    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-8vwpx\""
Jul 17 04:54:21.409400 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:21.409381    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-bn76n"]
Jul 17 04:54:21.415549 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:21.415523    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zw8z\" (UniqueName: \"kubernetes.io/projected/e9f30781-7638-4988-9e37-ee6f7d9b935e-kube-api-access-2zw8z\") pod \"cert-manager-55b4d954d5-bn76n\" (UID: \"e9f30781-7638-4988-9e37-ee6f7d9b935e\") " pod="cert-manager/cert-manager-55b4d954d5-bn76n"
Jul 17 04:54:21.415656 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:21.415556    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e9f30781-7638-4988-9e37-ee6f7d9b935e-bound-sa-token\") pod \"cert-manager-55b4d954d5-bn76n\" (UID: \"e9f30781-7638-4988-9e37-ee6f7d9b935e\") " pod="cert-manager/cert-manager-55b4d954d5-bn76n"
Jul 17 04:54:21.516845 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:21.516815    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zw8z\" (UniqueName: \"kubernetes.io/projected/e9f30781-7638-4988-9e37-ee6f7d9b935e-kube-api-access-2zw8z\") pod \"cert-manager-55b4d954d5-bn76n\" (UID: \"e9f30781-7638-4988-9e37-ee6f7d9b935e\") " pod="cert-manager/cert-manager-55b4d954d5-bn76n"
Jul 17 04:54:21.516845 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:21.516848    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e9f30781-7638-4988-9e37-ee6f7d9b935e-bound-sa-token\") pod \"cert-manager-55b4d954d5-bn76n\" (UID: \"e9f30781-7638-4988-9e37-ee6f7d9b935e\") " pod="cert-manager/cert-manager-55b4d954d5-bn76n"
Jul 17 04:54:21.525892 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:21.525868    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e9f30781-7638-4988-9e37-ee6f7d9b935e-bound-sa-token\") pod \"cert-manager-55b4d954d5-bn76n\" (UID: \"e9f30781-7638-4988-9e37-ee6f7d9b935e\") " pod="cert-manager/cert-manager-55b4d954d5-bn76n"
Jul 17 04:54:21.526094 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:21.526079    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zw8z\" (UniqueName: \"kubernetes.io/projected/e9f30781-7638-4988-9e37-ee6f7d9b935e-kube-api-access-2zw8z\") pod \"cert-manager-55b4d954d5-bn76n\" (UID: \"e9f30781-7638-4988-9e37-ee6f7d9b935e\") " pod="cert-manager/cert-manager-55b4d954d5-bn76n"
Jul 17 04:54:21.710114 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:21.710033    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-bn76n"
Jul 17 04:54:21.843695 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:21.843674    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-bn76n"]
Jul 17 04:54:21.845479 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:54:21.845452    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode9f30781_7638_4988_9e37_ee6f7d9b935e.slice/crio-4ad725057a7df4b495e9616493f3e4397626fc6ed78ae37ca3f020298c30efbb WatchSource:0}: Error finding container 4ad725057a7df4b495e9616493f3e4397626fc6ed78ae37ca3f020298c30efbb: Status 404 returned error can't find the container with id 4ad725057a7df4b495e9616493f3e4397626fc6ed78ae37ca3f020298c30efbb
Jul 17 04:54:22.082411 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:22.082377    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-bn76n" event={"ID":"e9f30781-7638-4988-9e37-ee6f7d9b935e","Type":"ContainerStarted","Data":"f6e0a05cd2f4c62999f4fdc9547352d22d364863292f7f469763e028bc076f12"}
Jul 17 04:54:22.082577 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:22.082421    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-bn76n" event={"ID":"e9f30781-7638-4988-9e37-ee6f7d9b935e","Type":"ContainerStarted","Data":"4ad725057a7df4b495e9616493f3e4397626fc6ed78ae37ca3f020298c30efbb"}
Jul 17 04:54:22.098235 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:22.098019    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-bn76n" podStartSLOduration=1.097999727 podStartE2EDuration="1.097999727s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 04:54:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 04:54:22.097797944 +0000 UTC m=+249.781391601" watchObservedRunningTime="2026-07-17 04:54:22.097999727 +0000 UTC m=+249.781593387"
Jul 17 04:54:47.020462 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:47.020375    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d857c78d5-ft2q8"]
Jul 17 04:54:47.024164 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:47.024141    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d857c78d5-ft2q8"
Jul 17 04:54:47.027528 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:47.027508    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 17 04:54:47.029824 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:47.029794    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 17 04:54:47.030197 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:47.030184    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-mx6ln\""
Jul 17 04:54:47.031539 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:47.031514    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 17 04:54:47.036267 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:47.036251    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 17 04:54:47.048615 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:47.048595    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d857c78d5-ft2q8"]
Jul 17 04:54:47.137727 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:47.137693    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a667f213-da2d-4290-b67c-700500da24b0-webhook-cert\") pod \"opendatahub-operator-controller-manager-d857c78d5-ft2q8\" (UID: \"a667f213-da2d-4290-b67c-700500da24b0\") " pod="opendatahub/opendatahub-operator-controller-manager-d857c78d5-ft2q8"
Jul 17 04:54:47.137934 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:47.137748    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24csg\" (UniqueName: \"kubernetes.io/projected/a667f213-da2d-4290-b67c-700500da24b0-kube-api-access-24csg\") pod \"opendatahub-operator-controller-manager-d857c78d5-ft2q8\" (UID: \"a667f213-da2d-4290-b67c-700500da24b0\") " pod="opendatahub/opendatahub-operator-controller-manager-d857c78d5-ft2q8"
Jul 17 04:54:47.137934 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:47.137848    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a667f213-da2d-4290-b67c-700500da24b0-tmp\") pod \"opendatahub-operator-controller-manager-d857c78d5-ft2q8\" (UID: \"a667f213-da2d-4290-b67c-700500da24b0\") " pod="opendatahub/opendatahub-operator-controller-manager-d857c78d5-ft2q8"
Jul 17 04:54:47.137934 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:47.137923    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a667f213-da2d-4290-b67c-700500da24b0-manifests\") pod \"opendatahub-operator-controller-manager-d857c78d5-ft2q8\" (UID: \"a667f213-da2d-4290-b67c-700500da24b0\") " pod="opendatahub/opendatahub-operator-controller-manager-d857c78d5-ft2q8"
Jul 17 04:54:47.138116 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:47.137952    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a667f213-da2d-4290-b67c-700500da24b0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d857c78d5-ft2q8\" (UID: \"a667f213-da2d-4290-b67c-700500da24b0\") " pod="opendatahub/opendatahub-operator-controller-manager-d857c78d5-ft2q8"
Jul 17 04:54:47.239058 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:47.239025    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24csg\" (UniqueName: \"kubernetes.io/projected/a667f213-da2d-4290-b67c-700500da24b0-kube-api-access-24csg\") pod \"opendatahub-operator-controller-manager-d857c78d5-ft2q8\" (UID: \"a667f213-da2d-4290-b67c-700500da24b0\") " pod="opendatahub/opendatahub-operator-controller-manager-d857c78d5-ft2q8"
Jul 17 04:54:47.239261 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:47.239098    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a667f213-da2d-4290-b67c-700500da24b0-tmp\") pod \"opendatahub-operator-controller-manager-d857c78d5-ft2q8\" (UID: \"a667f213-da2d-4290-b67c-700500da24b0\") " pod="opendatahub/opendatahub-operator-controller-manager-d857c78d5-ft2q8"
Jul 17 04:54:47.239261 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:47.239161    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a667f213-da2d-4290-b67c-700500da24b0-manifests\") pod \"opendatahub-operator-controller-manager-d857c78d5-ft2q8\" (UID: \"a667f213-da2d-4290-b67c-700500da24b0\") " pod="opendatahub/opendatahub-operator-controller-manager-d857c78d5-ft2q8"
Jul 17 04:54:47.239261 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:47.239187    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a667f213-da2d-4290-b67c-700500da24b0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d857c78d5-ft2q8\" (UID: \"a667f213-da2d-4290-b67c-700500da24b0\") " pod="opendatahub/opendatahub-operator-controller-manager-d857c78d5-ft2q8"
Jul 17 04:54:47.239261 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:47.239215    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a667f213-da2d-4290-b67c-700500da24b0-webhook-cert\") pod \"opendatahub-operator-controller-manager-d857c78d5-ft2q8\" (UID: \"a667f213-da2d-4290-b67c-700500da24b0\") " pod="opendatahub/opendatahub-operator-controller-manager-d857c78d5-ft2q8"
Jul 17 04:54:47.239574 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:47.239552    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a667f213-da2d-4290-b67c-700500da24b0-manifests\") pod \"opendatahub-operator-controller-manager-d857c78d5-ft2q8\" (UID: \"a667f213-da2d-4290-b67c-700500da24b0\") " pod="opendatahub/opendatahub-operator-controller-manager-d857c78d5-ft2q8"
Jul 17 04:54:47.241591 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:47.241568    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a667f213-da2d-4290-b67c-700500da24b0-tmp\") pod \"opendatahub-operator-controller-manager-d857c78d5-ft2q8\" (UID: \"a667f213-da2d-4290-b67c-700500da24b0\") " pod="opendatahub/opendatahub-operator-controller-manager-d857c78d5-ft2q8"
Jul 17 04:54:47.241718 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:47.241701    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a667f213-da2d-4290-b67c-700500da24b0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d857c78d5-ft2q8\" (UID: \"a667f213-da2d-4290-b67c-700500da24b0\") " pod="opendatahub/opendatahub-operator-controller-manager-d857c78d5-ft2q8"
Jul 17 04:54:47.241891 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:47.241869    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a667f213-da2d-4290-b67c-700500da24b0-webhook-cert\") pod \"opendatahub-operator-controller-manager-d857c78d5-ft2q8\" (UID: \"a667f213-da2d-4290-b67c-700500da24b0\") " pod="opendatahub/opendatahub-operator-controller-manager-d857c78d5-ft2q8"
Jul 17 04:54:47.250873 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:47.250765    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24csg\" (UniqueName: \"kubernetes.io/projected/a667f213-da2d-4290-b67c-700500da24b0-kube-api-access-24csg\") pod \"opendatahub-operator-controller-manager-d857c78d5-ft2q8\" (UID: \"a667f213-da2d-4290-b67c-700500da24b0\") " pod="opendatahub/opendatahub-operator-controller-manager-d857c78d5-ft2q8"
Jul 17 04:54:47.335039 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:47.334999    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d857c78d5-ft2q8"
Jul 17 04:54:47.474362 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:47.474337    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d857c78d5-ft2q8"]
Jul 17 04:54:47.477249 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:54:47.477219    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda667f213_da2d_4290_b67c_700500da24b0.slice/crio-99705ce1f6645467cce96ec56fd3a8c09612e4f8b04412b1b57c5cbc8ca8c7d5 WatchSource:0}: Error finding container 99705ce1f6645467cce96ec56fd3a8c09612e4f8b04412b1b57c5cbc8ca8c7d5: Status 404 returned error can't find the container with id 99705ce1f6645467cce96ec56fd3a8c09612e4f8b04412b1b57c5cbc8ca8c7d5
Jul 17 04:54:48.186276 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:48.186234    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d857c78d5-ft2q8" event={"ID":"a667f213-da2d-4290-b67c-700500da24b0","Type":"ContainerStarted","Data":"99705ce1f6645467cce96ec56fd3a8c09612e4f8b04412b1b57c5cbc8ca8c7d5"}
Jul 17 04:54:51.202104 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:51.202070    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d857c78d5-ft2q8" event={"ID":"a667f213-da2d-4290-b67c-700500da24b0","Type":"ContainerStarted","Data":"6e6c9652aa533ff7a2cf381b4a54318e3e49cff2180189abf9dbb2b459d49028"}
Jul 17 04:54:52.208375 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:52.208340    2562 generic.go:356] "Generic (PLEG): container finished" podID="a667f213-da2d-4290-b67c-700500da24b0" containerID="6e6c9652aa533ff7a2cf381b4a54318e3e49cff2180189abf9dbb2b459d49028" exitCode=0
Jul 17 04:54:52.208807 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:52.208430    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d857c78d5-ft2q8" event={"ID":"a667f213-da2d-4290-b67c-700500da24b0","Type":"ContainerDied","Data":"6e6c9652aa533ff7a2cf381b4a54318e3e49cff2180189abf9dbb2b459d49028"}
Jul 17 04:54:53.221030 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:53.220987    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d857c78d5-ft2q8" event={"ID":"a667f213-da2d-4290-b67c-700500da24b0","Type":"ContainerStarted","Data":"942a637aa5a478a590b31c4efd9036a5e1ac367338cfc7bbf3b4ae347507c008"}
Jul 17 04:54:53.221438 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:53.221123    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-d857c78d5-ft2q8"
Jul 17 04:54:53.244093 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:54:53.244045    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-d857c78d5-ft2q8" podStartSLOduration=2.770838448 podStartE2EDuration="7.244030796s" totalImagesPullingTime="3.376264119s" totalInitContainerRuntime="1.096928229s" isStatefulPod=false podCreationTimestamp="2026-07-17 04:54:46 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 04:54:53.24140757 +0000 UTC m=+280.925001244" watchObservedRunningTime="2026-07-17 04:54:53.244030796 +0000 UTC m=+280.927624509"
Jul 17 04:55:04.228215 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:04.228179    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-d857c78d5-ft2q8"
Jul 17 04:55:12.716329 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:12.716298    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tx9gj_c1baeee2-350c-43c5-9a04-fb34378edb5c/ovn-acl-logging/0.log"
Jul 17 04:55:12.723230 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:12.723211    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tx9gj_c1baeee2-350c-43c5-9a04-fb34378edb5c/ovn-acl-logging/0.log"
Jul 17 04:55:12.723376 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:12.723308    2562 kubelet.go:1648] "Image garbage collection succeeded"
Jul 17 04:55:14.553386 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.553350    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9tb57"]
Jul 17 04:55:14.555619 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.555604    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9tb57"
Jul 17 04:55:14.559283 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.559263    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 17 04:55:14.559413 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.559263    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 17 04:55:14.559510 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.559491    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 17 04:55:14.559782 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.559759    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 17 04:55:14.560085 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.560067    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 17 04:55:14.560299 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.560282    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 17 04:55:14.562554 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.562534    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-bk6rk\""
Jul 17 04:55:14.580366 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.580342    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9tb57"]
Jul 17 04:55:14.583040 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.583010    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0a7089b-2cde-439a-8cd8-187338f02f18-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-9tb57\" (UID: \"e0a7089b-2cde-439a-8cd8-187338f02f18\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9tb57"
Jul 17 04:55:14.583136 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.583079    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e0a7089b-2cde-439a-8cd8-187338f02f18-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-9tb57\" (UID: \"e0a7089b-2cde-439a-8cd8-187338f02f18\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9tb57"
Jul 17 04:55:14.583175 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.583154    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0a7089b-2cde-439a-8cd8-187338f02f18-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-9tb57\" (UID: \"e0a7089b-2cde-439a-8cd8-187338f02f18\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9tb57"
Jul 17 04:55:14.583240 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.583220    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e0a7089b-2cde-439a-8cd8-187338f02f18-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-9tb57\" (UID: \"e0a7089b-2cde-439a-8cd8-187338f02f18\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9tb57"
Jul 17 04:55:14.583305 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.583287    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfck8\" (UniqueName: \"kubernetes.io/projected/e0a7089b-2cde-439a-8cd8-187338f02f18-kube-api-access-bfck8\") pod \"istiod-openshift-gateway-94bb8fbfd-9tb57\" (UID: \"e0a7089b-2cde-439a-8cd8-187338f02f18\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9tb57"
Jul 17 04:55:14.583339 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.583324    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e0a7089b-2cde-439a-8cd8-187338f02f18-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-9tb57\" (UID: \"e0a7089b-2cde-439a-8cd8-187338f02f18\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9tb57"
Jul 17 04:55:14.583378 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.583364    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e0a7089b-2cde-439a-8cd8-187338f02f18-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-9tb57\" (UID: \"e0a7089b-2cde-439a-8cd8-187338f02f18\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9tb57"
Jul 17 04:55:14.677891 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.677852    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-559bdd8b4c-gj9kt"]
Jul 17 04:55:14.680631 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.680608    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-559bdd8b4c-gj9kt"
Jul 17 04:55:14.683649 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.683628    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 17 04:55:14.683880 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.683851    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfck8\" (UniqueName: \"kubernetes.io/projected/e0a7089b-2cde-439a-8cd8-187338f02f18-kube-api-access-bfck8\") pod \"istiod-openshift-gateway-94bb8fbfd-9tb57\" (UID: \"e0a7089b-2cde-439a-8cd8-187338f02f18\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9tb57"
Jul 17 04:55:14.684011 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.683893    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e0a7089b-2cde-439a-8cd8-187338f02f18-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-9tb57\" (UID: \"e0a7089b-2cde-439a-8cd8-187338f02f18\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9tb57"
Jul 17 04:55:14.684011 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.683933    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e0a7089b-2cde-439a-8cd8-187338f02f18-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-9tb57\" (UID: \"e0a7089b-2cde-439a-8cd8-187338f02f18\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9tb57"
Jul 17 04:55:14.684011 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.683632    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 17 04:55:14.684011 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.683642    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 17 04:55:14.684242 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.684022    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0a7089b-2cde-439a-8cd8-187338f02f18-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-9tb57\" (UID: \"e0a7089b-2cde-439a-8cd8-187338f02f18\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9tb57"
Jul 17 04:55:14.684242 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.683642    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 17 04:55:14.684242 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.684073    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e0a7089b-2cde-439a-8cd8-187338f02f18-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-9tb57\" (UID: \"e0a7089b-2cde-439a-8cd8-187338f02f18\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9tb57"
Jul 17 04:55:14.684242 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.683632    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 17 04:55:14.684242 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.684128    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0a7089b-2cde-439a-8cd8-187338f02f18-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-9tb57\" (UID: \"e0a7089b-2cde-439a-8cd8-187338f02f18\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9tb57"
Jul 17 04:55:14.684242 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.683665    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-k8t6r\""
Jul 17 04:55:14.684242 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.684166    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e0a7089b-2cde-439a-8cd8-187338f02f18-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-9tb57\" (UID: \"e0a7089b-2cde-439a-8cd8-187338f02f18\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9tb57"
Jul 17 04:55:14.684632 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.684607    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e0a7089b-2cde-439a-8cd8-187338f02f18-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-9tb57\" (UID: \"e0a7089b-2cde-439a-8cd8-187338f02f18\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9tb57"
Jul 17 04:55:14.686873 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.686853    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0a7089b-2cde-439a-8cd8-187338f02f18-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-9tb57\" (UID: \"e0a7089b-2cde-439a-8cd8-187338f02f18\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9tb57"
Jul 17 04:55:14.687171 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.687152    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e0a7089b-2cde-439a-8cd8-187338f02f18-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-9tb57\" (UID: \"e0a7089b-2cde-439a-8cd8-187338f02f18\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9tb57"
Jul 17 04:55:14.687230 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.687173    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e0a7089b-2cde-439a-8cd8-187338f02f18-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-9tb57\" (UID: \"e0a7089b-2cde-439a-8cd8-187338f02f18\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9tb57"
Jul 17 04:55:14.687275 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.687255    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e0a7089b-2cde-439a-8cd8-187338f02f18-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-9tb57\" (UID: \"e0a7089b-2cde-439a-8cd8-187338f02f18\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9tb57"
Jul 17 04:55:14.690765 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.690737    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-559bdd8b4c-gj9kt"]
Jul 17 04:55:14.704490 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.704466    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0a7089b-2cde-439a-8cd8-187338f02f18-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-9tb57\" (UID: \"e0a7089b-2cde-439a-8cd8-187338f02f18\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9tb57"
Jul 17 04:55:14.704886 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.704867    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfck8\" (UniqueName: \"kubernetes.io/projected/e0a7089b-2cde-439a-8cd8-187338f02f18-kube-api-access-bfck8\") pod \"istiod-openshift-gateway-94bb8fbfd-9tb57\" (UID: \"e0a7089b-2cde-439a-8cd8-187338f02f18\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9tb57"
Jul 17 04:55:14.784984 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.784933    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a242308f-aa18-435e-a83c-3eb68deff925-manager-config\") pod \"lws-controller-manager-559bdd8b4c-gj9kt\" (UID: \"a242308f-aa18-435e-a83c-3eb68deff925\") " pod="openshift-lws-operator/lws-controller-manager-559bdd8b4c-gj9kt"
Jul 17 04:55:14.784984 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.784991    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a242308f-aa18-435e-a83c-3eb68deff925-metrics-cert\") pod \"lws-controller-manager-559bdd8b4c-gj9kt\" (UID: \"a242308f-aa18-435e-a83c-3eb68deff925\") " pod="openshift-lws-operator/lws-controller-manager-559bdd8b4c-gj9kt"
Jul 17 04:55:14.785202 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.785071    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhwp7\" (UniqueName: \"kubernetes.io/projected/a242308f-aa18-435e-a83c-3eb68deff925-kube-api-access-fhwp7\") pod \"lws-controller-manager-559bdd8b4c-gj9kt\" (UID: \"a242308f-aa18-435e-a83c-3eb68deff925\") " pod="openshift-lws-operator/lws-controller-manager-559bdd8b4c-gj9kt"
Jul 17 04:55:14.785202 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.785115    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a242308f-aa18-435e-a83c-3eb68deff925-cert\") pod \"lws-controller-manager-559bdd8b4c-gj9kt\" (UID: \"a242308f-aa18-435e-a83c-3eb68deff925\") " pod="openshift-lws-operator/lws-controller-manager-559bdd8b4c-gj9kt"
Jul 17 04:55:14.865221 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.865181    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9tb57"
Jul 17 04:55:14.885902 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.885869    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a242308f-aa18-435e-a83c-3eb68deff925-cert\") pod \"lws-controller-manager-559bdd8b4c-gj9kt\" (UID: \"a242308f-aa18-435e-a83c-3eb68deff925\") " pod="openshift-lws-operator/lws-controller-manager-559bdd8b4c-gj9kt"
Jul 17 04:55:14.886073 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.886017    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a242308f-aa18-435e-a83c-3eb68deff925-manager-config\") pod \"lws-controller-manager-559bdd8b4c-gj9kt\" (UID: \"a242308f-aa18-435e-a83c-3eb68deff925\") " pod="openshift-lws-operator/lws-controller-manager-559bdd8b4c-gj9kt"
Jul 17 04:55:14.886073 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.886044    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a242308f-aa18-435e-a83c-3eb68deff925-metrics-cert\") pod \"lws-controller-manager-559bdd8b4c-gj9kt\" (UID: \"a242308f-aa18-435e-a83c-3eb68deff925\") " pod="openshift-lws-operator/lws-controller-manager-559bdd8b4c-gj9kt"
Jul 17 04:55:14.886205 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.886173    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhwp7\" (UniqueName: \"kubernetes.io/projected/a242308f-aa18-435e-a83c-3eb68deff925-kube-api-access-fhwp7\") pod \"lws-controller-manager-559bdd8b4c-gj9kt\" (UID: \"a242308f-aa18-435e-a83c-3eb68deff925\") " pod="openshift-lws-operator/lws-controller-manager-559bdd8b4c-gj9kt"
Jul 17 04:55:14.887022 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.886904    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a242308f-aa18-435e-a83c-3eb68deff925-manager-config\") pod \"lws-controller-manager-559bdd8b4c-gj9kt\" (UID: \"a242308f-aa18-435e-a83c-3eb68deff925\") " pod="openshift-lws-operator/lws-controller-manager-559bdd8b4c-gj9kt"
Jul 17 04:55:14.889250 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.889227    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a242308f-aa18-435e-a83c-3eb68deff925-cert\") pod \"lws-controller-manager-559bdd8b4c-gj9kt\" (UID: \"a242308f-aa18-435e-a83c-3eb68deff925\") " pod="openshift-lws-operator/lws-controller-manager-559bdd8b4c-gj9kt"
Jul 17 04:55:14.889856 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.889822    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a242308f-aa18-435e-a83c-3eb68deff925-metrics-cert\") pod \"lws-controller-manager-559bdd8b4c-gj9kt\" (UID: \"a242308f-aa18-435e-a83c-3eb68deff925\") " pod="openshift-lws-operator/lws-controller-manager-559bdd8b4c-gj9kt"
Jul 17 04:55:14.896047 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:14.896020    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhwp7\" (UniqueName: \"kubernetes.io/projected/a242308f-aa18-435e-a83c-3eb68deff925-kube-api-access-fhwp7\") pod \"lws-controller-manager-559bdd8b4c-gj9kt\" (UID: \"a242308f-aa18-435e-a83c-3eb68deff925\") " pod="openshift-lws-operator/lws-controller-manager-559bdd8b4c-gj9kt"
Jul 17 04:55:15.007142 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:15.007104    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-559bdd8b4c-gj9kt"
Jul 17 04:55:15.032825 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:15.032801    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9tb57"]
Jul 17 04:55:15.034877 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:55:15.034842    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0a7089b_2cde_439a_8cd8_187338f02f18.slice/crio-4cedb69024b20567725f3be036662e95e84e250b6d61eda7e7c9bdf9036bd7c6 WatchSource:0}: Error finding container 4cedb69024b20567725f3be036662e95e84e250b6d61eda7e7c9bdf9036bd7c6: Status 404 returned error can't find the container with id 4cedb69024b20567725f3be036662e95e84e250b6d61eda7e7c9bdf9036bd7c6
Jul 17 04:55:15.037147 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:15.037126    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 04:55:15.141259 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:15.141234    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-559bdd8b4c-gj9kt"]
Jul 17 04:55:15.143171 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:55:15.143148    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda242308f_aa18_435e_a83c_3eb68deff925.slice/crio-da95a1d9cdc9ef28c16125645139a22c35efcb31ae70eebac5ec73a674abf16f WatchSource:0}: Error finding container da95a1d9cdc9ef28c16125645139a22c35efcb31ae70eebac5ec73a674abf16f: Status 404 returned error can't find the container with id da95a1d9cdc9ef28c16125645139a22c35efcb31ae70eebac5ec73a674abf16f
Jul 17 04:55:15.320844 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:15.320810    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-559bdd8b4c-gj9kt" event={"ID":"a242308f-aa18-435e-a83c-3eb68deff925","Type":"ContainerStarted","Data":"da95a1d9cdc9ef28c16125645139a22c35efcb31ae70eebac5ec73a674abf16f"}
Jul 17 04:55:15.322166 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:15.322146    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9tb57" event={"ID":"e0a7089b-2cde-439a-8cd8-187338f02f18","Type":"ContainerStarted","Data":"4cedb69024b20567725f3be036662e95e84e250b6d61eda7e7c9bdf9036bd7c6"}
Jul 17 04:55:18.610929 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:18.610891    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 04:55:18.611240 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:18.610958    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 04:55:19.347947 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:19.347906    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9tb57" event={"ID":"e0a7089b-2cde-439a-8cd8-187338f02f18","Type":"ContainerStarted","Data":"23d9a185d3def08db80e970b54310e363ccdeacd04b3d89fe9e6f748dd4fa4df"}
Jul 17 04:55:19.348197 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:19.348173    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9tb57"
Jul 17 04:55:19.349746 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:19.349710    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-559bdd8b4c-gj9kt" event={"ID":"a242308f-aa18-435e-a83c-3eb68deff925","Type":"ContainerStarted","Data":"365c7457f8af454768f405bd6617eb518c55f8dd0b5a39dda9b5cbca6236a833"}
Jul 17 04:55:19.349874 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:19.349771    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-559bdd8b4c-gj9kt"
Jul 17 04:55:19.350232 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:19.350204    2562 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-9tb57 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 17 04:55:19.350330 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:19.350251    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9tb57" podUID="e0a7089b-2cde-439a-8cd8-187338f02f18" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 17 04:55:19.373311 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:19.373255    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9tb57" podStartSLOduration=1.799830301 podStartE2EDuration="5.373236346s" totalImagesPullingTime="3.573406045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 04:55:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 04:55:19.371880884 +0000 UTC m=+307.055474538" watchObservedRunningTime="2026-07-17 04:55:19.373236346 +0000 UTC m=+307.056830007"
Jul 17 04:55:19.391057 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:19.391008    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-559bdd8b4c-gj9kt" podStartSLOduration=1.925939637 podStartE2EDuration="5.390993479s" totalImagesPullingTime="3.465053842s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 04:55:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 04:55:19.38906784 +0000 UTC m=+307.072661479" watchObservedRunningTime="2026-07-17 04:55:19.390993479 +0000 UTC m=+307.074587136"
Jul 17 04:55:20.354854 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:20.354820    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9tb57"
Jul 17 04:55:30.356495 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:55:30.356461    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-559bdd8b4c-gj9kt"
Jul 17 04:57:11.189844 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:11.189791    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-86wtr"]
Jul 17 04:57:11.192000 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:11.191983    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-86wtr"
Jul 17 04:57:11.194870 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:11.194848    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 17 04:57:11.194984 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:11.194926    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 17 04:57:11.194984 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:11.194957    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-dznhn\""
Jul 17 04:57:11.204945 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:11.204921    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-86wtr"]
Jul 17 04:57:11.282848 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:11.282816    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fa712e7f-2d76-4305-8247-809b9ea3ca2d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-86wtr\" (UID: \"fa712e7f-2d76-4305-8247-809b9ea3ca2d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-86wtr"
Jul 17 04:57:11.282848 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:11.282847    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sj79s\" (UniqueName: \"kubernetes.io/projected/fa712e7f-2d76-4305-8247-809b9ea3ca2d-kube-api-access-sj79s\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-86wtr\" (UID: \"fa712e7f-2d76-4305-8247-809b9ea3ca2d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-86wtr"
Jul 17 04:57:11.383929 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:11.383900    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fa712e7f-2d76-4305-8247-809b9ea3ca2d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-86wtr\" (UID: \"fa712e7f-2d76-4305-8247-809b9ea3ca2d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-86wtr"
Jul 17 04:57:11.383929 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:11.383931    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sj79s\" (UniqueName: \"kubernetes.io/projected/fa712e7f-2d76-4305-8247-809b9ea3ca2d-kube-api-access-sj79s\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-86wtr\" (UID: \"fa712e7f-2d76-4305-8247-809b9ea3ca2d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-86wtr"
Jul 17 04:57:11.384292 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:11.384273    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fa712e7f-2d76-4305-8247-809b9ea3ca2d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-86wtr\" (UID: \"fa712e7f-2d76-4305-8247-809b9ea3ca2d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-86wtr"
Jul 17 04:57:11.392579 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:11.392552    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sj79s\" (UniqueName: \"kubernetes.io/projected/fa712e7f-2d76-4305-8247-809b9ea3ca2d-kube-api-access-sj79s\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-86wtr\" (UID: \"fa712e7f-2d76-4305-8247-809b9ea3ca2d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-86wtr"
Jul 17 04:57:11.502325 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:11.502246    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-86wtr"
Jul 17 04:57:11.632406 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:11.632380    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-86wtr"]
Jul 17 04:57:11.635478 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:57:11.635436    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa712e7f_2d76_4305_8247_809b9ea3ca2d.slice/crio-dd8958aca0283dc47b278f3e1d209dfac996b3610ee753c6d1cba8a2a4b22d12 WatchSource:0}: Error finding container dd8958aca0283dc47b278f3e1d209dfac996b3610ee753c6d1cba8a2a4b22d12: Status 404 returned error can't find the container with id dd8958aca0283dc47b278f3e1d209dfac996b3610ee753c6d1cba8a2a4b22d12
Jul 17 04:57:11.859571 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:11.859536    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-86wtr" event={"ID":"fa712e7f-2d76-4305-8247-809b9ea3ca2d","Type":"ContainerStarted","Data":"dd8958aca0283dc47b278f3e1d209dfac996b3610ee753c6d1cba8a2a4b22d12"}
Jul 17 04:57:11.899766 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:11.899733    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-86wtr"]
Jul 17 04:57:11.905552 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:11.905528    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-86wtr"]
Jul 17 04:57:11.924254 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:11.924227    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2k8nq"]
Jul 17 04:57:11.927535 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:11.927514    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2k8nq"
Jul 17 04:57:11.940488 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:11.940458    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2k8nq"]
Jul 17 04:57:11.990020 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:11.989987    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e837d21-a0e1-4565-908d-51f9e0237db2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-2k8nq\" (UID: \"7e837d21-a0e1-4565-908d-51f9e0237db2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2k8nq"
Jul 17 04:57:11.990206 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:11.990071    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9wqk\" (UniqueName: \"kubernetes.io/projected/7e837d21-a0e1-4565-908d-51f9e0237db2-kube-api-access-x9wqk\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-2k8nq\" (UID: \"7e837d21-a0e1-4565-908d-51f9e0237db2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2k8nq"
Jul 17 04:57:12.090953 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:12.090911    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e837d21-a0e1-4565-908d-51f9e0237db2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-2k8nq\" (UID: \"7e837d21-a0e1-4565-908d-51f9e0237db2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2k8nq"
Jul 17 04:57:12.091163 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:12.091012    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9wqk\" (UniqueName: \"kubernetes.io/projected/7e837d21-a0e1-4565-908d-51f9e0237db2-kube-api-access-x9wqk\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-2k8nq\" (UID: \"7e837d21-a0e1-4565-908d-51f9e0237db2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2k8nq"
Jul 17 04:57:12.091306 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:12.091283    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e837d21-a0e1-4565-908d-51f9e0237db2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-2k8nq\" (UID: \"7e837d21-a0e1-4565-908d-51f9e0237db2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2k8nq"
Jul 17 04:57:12.100090 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:12.100055    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9wqk\" (UniqueName: \"kubernetes.io/projected/7e837d21-a0e1-4565-908d-51f9e0237db2-kube-api-access-x9wqk\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-2k8nq\" (UID: \"7e837d21-a0e1-4565-908d-51f9e0237db2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2k8nq"
Jul 17 04:57:12.240434 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:12.240353    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2k8nq"
Jul 17 04:57:12.411442 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:12.411415    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2k8nq"]
Jul 17 04:57:12.428384 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:57:12.428357    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e837d21_a0e1_4565_908d_51f9e0237db2.slice/crio-5a954656abf207944f1ee0496b43fe792fde129035a88bc718084476e0090359 WatchSource:0}: Error finding container 5a954656abf207944f1ee0496b43fe792fde129035a88bc718084476e0090359: Status 404 returned error can't find the container with id 5a954656abf207944f1ee0496b43fe792fde129035a88bc718084476e0090359
Jul 17 04:57:12.866995 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:12.866940    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2k8nq" event={"ID":"7e837d21-a0e1-4565-908d-51f9e0237db2","Type":"ContainerStarted","Data":"5a954656abf207944f1ee0496b43fe792fde129035a88bc718084476e0090359"}
Jul 17 04:57:16.888857 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:16.888798    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2k8nq" event={"ID":"7e837d21-a0e1-4565-908d-51f9e0237db2","Type":"ContainerStarted","Data":"8028e941bc23ed7cc7fc155b1f014577d7fbe9a7fcb7c18aaaf9d0ddee5b32be"}
Jul 17 04:57:16.889348 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:16.888941    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2k8nq"
Jul 17 04:57:16.890514 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:16.890485    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-86wtr" podUID="fa712e7f-2d76-4305-8247-809b9ea3ca2d" containerName="manager" containerID="cri-o://08350cec99489b047d4a4b3ce044e69ddcdf2e750eb47a88010ddb1ff28be7f0" gracePeriod=2
Jul 17 04:57:16.908152 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:16.908059    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2k8nq" podStartSLOduration=2.320517362 podStartE2EDuration="5.908039466s" totalImagesPullingTime="3.587522104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 04:57:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 04:57:16.905809919 +0000 UTC m=+424.589403576" watchObservedRunningTime="2026-07-17 04:57:16.908039466 +0000 UTC m=+424.591633124"
Jul 17 04:57:17.127276 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:17.127249    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-86wtr"
Jul 17 04:57:17.129627 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:57:17.129602    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-86wtr\" is forbidden: User \"system:node:ip-10-0-132-19.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-19.ec2.internal' and this object" podUID="fa712e7f-2d76-4305-8247-809b9ea3ca2d" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-86wtr"
Jul 17 04:57:17.240803 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:17.240716    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fa712e7f-2d76-4305-8247-809b9ea3ca2d-extensions-socket-volume\") pod \"fa712e7f-2d76-4305-8247-809b9ea3ca2d\" (UID: \"fa712e7f-2d76-4305-8247-809b9ea3ca2d\") "
Jul 17 04:57:17.240803 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:17.240770    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sj79s\" (UniqueName: \"kubernetes.io/projected/fa712e7f-2d76-4305-8247-809b9ea3ca2d-kube-api-access-sj79s\") pod \"fa712e7f-2d76-4305-8247-809b9ea3ca2d\" (UID: \"fa712e7f-2d76-4305-8247-809b9ea3ca2d\") "
Jul 17 04:57:17.241040 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:17.241011    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa712e7f-2d76-4305-8247-809b9ea3ca2d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "fa712e7f-2d76-4305-8247-809b9ea3ca2d" (UID: "fa712e7f-2d76-4305-8247-809b9ea3ca2d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 04:57:17.243144 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:17.243119    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa712e7f-2d76-4305-8247-809b9ea3ca2d-kube-api-access-sj79s" (OuterVolumeSpecName: "kube-api-access-sj79s") pod "fa712e7f-2d76-4305-8247-809b9ea3ca2d" (UID: "fa712e7f-2d76-4305-8247-809b9ea3ca2d"). InnerVolumeSpecName "kube-api-access-sj79s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 04:57:17.341770 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:17.341722    2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fa712e7f-2d76-4305-8247-809b9ea3ca2d-extensions-socket-volume\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 04:57:17.341770 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:17.341761    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sj79s\" (UniqueName: \"kubernetes.io/projected/fa712e7f-2d76-4305-8247-809b9ea3ca2d-kube-api-access-sj79s\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 04:57:17.897197 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:17.897156    2562 generic.go:356] "Generic (PLEG): container finished" podID="fa712e7f-2d76-4305-8247-809b9ea3ca2d" containerID="08350cec99489b047d4a4b3ce044e69ddcdf2e750eb47a88010ddb1ff28be7f0" exitCode=2
Jul 17 04:57:17.897197 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:17.897201    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-86wtr"
Jul 17 04:57:17.897651 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:17.897257    2562 scope.go:117] "RemoveContainer" containerID="08350cec99489b047d4a4b3ce044e69ddcdf2e750eb47a88010ddb1ff28be7f0"
Jul 17 04:57:17.899510 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:57:17.899485    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-86wtr\" is forbidden: User \"system:node:ip-10-0-132-19.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-19.ec2.internal' and this object" podUID="fa712e7f-2d76-4305-8247-809b9ea3ca2d" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-86wtr"
Jul 17 04:57:17.906545 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:17.906526    2562 scope.go:117] "RemoveContainer" containerID="08350cec99489b047d4a4b3ce044e69ddcdf2e750eb47a88010ddb1ff28be7f0"
Jul 17 04:57:17.906825 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:57:17.906807    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08350cec99489b047d4a4b3ce044e69ddcdf2e750eb47a88010ddb1ff28be7f0\": container with ID starting with 08350cec99489b047d4a4b3ce044e69ddcdf2e750eb47a88010ddb1ff28be7f0 not found: ID does not exist" containerID="08350cec99489b047d4a4b3ce044e69ddcdf2e750eb47a88010ddb1ff28be7f0"
Jul 17 04:57:17.906876 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:17.906834    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08350cec99489b047d4a4b3ce044e69ddcdf2e750eb47a88010ddb1ff28be7f0"} err="failed to get container status \"08350cec99489b047d4a4b3ce044e69ddcdf2e750eb47a88010ddb1ff28be7f0\": rpc error: code = NotFound desc = could not find container \"08350cec99489b047d4a4b3ce044e69ddcdf2e750eb47a88010ddb1ff28be7f0\": container with ID starting with 08350cec99489b047d4a4b3ce044e69ddcdf2e750eb47a88010ddb1ff28be7f0 not found: ID does not exist"
Jul 17 04:57:17.907190 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:57:17.907167    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-86wtr\" is forbidden: User \"system:node:ip-10-0-132-19.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-19.ec2.internal' and this object" podUID="fa712e7f-2d76-4305-8247-809b9ea3ca2d" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-86wtr"
Jul 17 04:57:18.884361 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:18.884326    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa712e7f-2d76-4305-8247-809b9ea3ca2d" path="/var/lib/kubelet/pods/fa712e7f-2d76-4305-8247-809b9ea3ca2d/volumes"
Jul 17 04:57:27.900135 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:27.900096    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2k8nq"
Jul 17 04:57:40.483624 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:40.483546    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2k8nq"]
Jul 17 04:57:40.484134 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:40.483737    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2k8nq" podUID="7e837d21-a0e1-4565-908d-51f9e0237db2" containerName="manager" containerID="cri-o://8028e941bc23ed7cc7fc155b1f014577d7fbe9a7fcb7c18aaaf9d0ddee5b32be" gracePeriod=10
Jul 17 04:57:40.734153 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:40.734098    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2k8nq"
Jul 17 04:57:40.748610 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:40.748585    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e837d21-a0e1-4565-908d-51f9e0237db2-extensions-socket-volume\") pod \"7e837d21-a0e1-4565-908d-51f9e0237db2\" (UID: \"7e837d21-a0e1-4565-908d-51f9e0237db2\") "
Jul 17 04:57:40.748729 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:40.748650    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x9wqk\" (UniqueName: \"kubernetes.io/projected/7e837d21-a0e1-4565-908d-51f9e0237db2-kube-api-access-x9wqk\") pod \"7e837d21-a0e1-4565-908d-51f9e0237db2\" (UID: \"7e837d21-a0e1-4565-908d-51f9e0237db2\") "
Jul 17 04:57:40.749110 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:40.749090    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e837d21-a0e1-4565-908d-51f9e0237db2-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7e837d21-a0e1-4565-908d-51f9e0237db2" (UID: "7e837d21-a0e1-4565-908d-51f9e0237db2"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 04:57:40.750796 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:40.750773    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e837d21-a0e1-4565-908d-51f9e0237db2-kube-api-access-x9wqk" (OuterVolumeSpecName: "kube-api-access-x9wqk") pod "7e837d21-a0e1-4565-908d-51f9e0237db2" (UID: "7e837d21-a0e1-4565-908d-51f9e0237db2"). InnerVolumeSpecName "kube-api-access-x9wqk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 04:57:40.850165 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:40.850136    2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e837d21-a0e1-4565-908d-51f9e0237db2-extensions-socket-volume\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 04:57:40.850165 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:40.850160    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9wqk\" (UniqueName: \"kubernetes.io/projected/7e837d21-a0e1-4565-908d-51f9e0237db2-kube-api-access-x9wqk\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 04:57:41.004378 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:41.004295    2562 generic.go:356] "Generic (PLEG): container finished" podID="7e837d21-a0e1-4565-908d-51f9e0237db2" containerID="8028e941bc23ed7cc7fc155b1f014577d7fbe9a7fcb7c18aaaf9d0ddee5b32be" exitCode=0
Jul 17 04:57:41.004378 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:41.004361    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2k8nq"
Jul 17 04:57:41.004537 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:41.004360    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2k8nq" event={"ID":"7e837d21-a0e1-4565-908d-51f9e0237db2","Type":"ContainerDied","Data":"8028e941bc23ed7cc7fc155b1f014577d7fbe9a7fcb7c18aaaf9d0ddee5b32be"}
Jul 17 04:57:41.004537 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:41.004481    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2k8nq" event={"ID":"7e837d21-a0e1-4565-908d-51f9e0237db2","Type":"ContainerDied","Data":"5a954656abf207944f1ee0496b43fe792fde129035a88bc718084476e0090359"}
Jul 17 04:57:41.004537 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:41.004504    2562 scope.go:117] "RemoveContainer" containerID="8028e941bc23ed7cc7fc155b1f014577d7fbe9a7fcb7c18aaaf9d0ddee5b32be"
Jul 17 04:57:41.012817 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:41.012796    2562 scope.go:117] "RemoveContainer" containerID="8028e941bc23ed7cc7fc155b1f014577d7fbe9a7fcb7c18aaaf9d0ddee5b32be"
Jul 17 04:57:41.013096 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:57:41.013077    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8028e941bc23ed7cc7fc155b1f014577d7fbe9a7fcb7c18aaaf9d0ddee5b32be\": container with ID starting with 8028e941bc23ed7cc7fc155b1f014577d7fbe9a7fcb7c18aaaf9d0ddee5b32be not found: ID does not exist" containerID="8028e941bc23ed7cc7fc155b1f014577d7fbe9a7fcb7c18aaaf9d0ddee5b32be"
Jul 17 04:57:41.013148 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:41.013104    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8028e941bc23ed7cc7fc155b1f014577d7fbe9a7fcb7c18aaaf9d0ddee5b32be"} err="failed to get container status \"8028e941bc23ed7cc7fc155b1f014577d7fbe9a7fcb7c18aaaf9d0ddee5b32be\": rpc error: code = NotFound desc = could not find container \"8028e941bc23ed7cc7fc155b1f014577d7fbe9a7fcb7c18aaaf9d0ddee5b32be\": container with ID starting with 8028e941bc23ed7cc7fc155b1f014577d7fbe9a7fcb7c18aaaf9d0ddee5b32be not found: ID does not exist"
Jul 17 04:57:41.023284 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:41.023257    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2k8nq"]
Jul 17 04:57:41.027166 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:41.027146    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-2k8nq"]
Jul 17 04:57:42.884416 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:42.884380    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e837d21-a0e1-4565-908d-51f9e0237db2" path="/var/lib/kubelet/pods/7e837d21-a0e1-4565-908d-51f9e0237db2/volumes"
Jul 17 04:57:44.233479 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:44.233450    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx"]
Jul 17 04:57:44.233830 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:44.233821    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa712e7f-2d76-4305-8247-809b9ea3ca2d" containerName="manager"
Jul 17 04:57:44.233870 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:44.233832    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa712e7f-2d76-4305-8247-809b9ea3ca2d" containerName="manager"
Jul 17 04:57:44.233870 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:44.233855    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e837d21-a0e1-4565-908d-51f9e0237db2" containerName="manager"
Jul 17 04:57:44.233870 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:44.233861    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e837d21-a0e1-4565-908d-51f9e0237db2" containerName="manager"
Jul 17 04:57:44.233957 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:44.233916    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e837d21-a0e1-4565-908d-51f9e0237db2" containerName="manager"
Jul 17 04:57:44.233957 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:44.233924    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa712e7f-2d76-4305-8247-809b9ea3ca2d" containerName="manager"
Jul 17 04:57:44.240553 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:44.240534    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx"
Jul 17 04:57:44.246209 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:44.246189    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-2lk22\""
Jul 17 04:57:44.263629 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:44.263597    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx"]
Jul 17 04:57:44.277500 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:44.277474    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/162f43aa-9c5c-4148-8f41-4c1f43c9d7a9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-k2snx\" (UID: \"162f43aa-9c5c-4148-8f41-4c1f43c9d7a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx"
Jul 17 04:57:44.277652 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:44.277514    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/162f43aa-9c5c-4148-8f41-4c1f43c9d7a9-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-k2snx\" (UID: \"162f43aa-9c5c-4148-8f41-4c1f43c9d7a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx"
Jul 17 04:57:44.277652 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:44.277534    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/162f43aa-9c5c-4148-8f41-4c1f43c9d7a9-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-k2snx\" (UID: \"162f43aa-9c5c-4148-8f41-4c1f43c9d7a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx"
Jul 17 04:57:44.277652 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:44.277564    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/162f43aa-9c5c-4148-8f41-4c1f43c9d7a9-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-k2snx\" (UID: \"162f43aa-9c5c-4148-8f41-4c1f43c9d7a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx"
Jul 17 04:57:44.277652 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:44.277626    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/162f43aa-9c5c-4148-8f41-4c1f43c9d7a9-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-k2snx\" (UID: \"162f43aa-9c5c-4148-8f41-4c1f43c9d7a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx"
Jul 17 04:57:44.277652 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:44.277650    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/162f43aa-9c5c-4148-8f41-4c1f43c9d7a9-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-k2snx\" (UID: \"162f43aa-9c5c-4148-8f41-4c1f43c9d7a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx"
Jul 17 04:57:44.277895 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:44.277667    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/162f43aa-9c5c-4148-8f41-4c1f43c9d7a9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-k2snx\" (UID: \"162f43aa-9c5c-4148-8f41-4c1f43c9d7a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx"
Jul 17 04:57:44.277895 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:44.277740    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qx9bk\" (UniqueName: \"kubernetes.io/projected/162f43aa-9c5c-4148-8f41-4c1f43c9d7a9-kube-api-access-qx9bk\") pod \"maas-default-gateway-openshift-default-5896d4dd49-k2snx\" (UID: \"162f43aa-9c5c-4148-8f41-4c1f43c9d7a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx"
Jul 17 04:57:44.277895 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:44.277791    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/162f43aa-9c5c-4148-8f41-4c1f43c9d7a9-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-k2snx\" (UID: \"162f43aa-9c5c-4148-8f41-4c1f43c9d7a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx"
Jul 17 04:57:44.379196 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:44.379165    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/162f43aa-9c5c-4148-8f41-4c1f43c9d7a9-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-k2snx\" (UID: \"162f43aa-9c5c-4148-8f41-4c1f43c9d7a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx"
Jul 17 04:57:44.379371 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:44.379205    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/162f43aa-9c5c-4148-8f41-4c1f43c9d7a9-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-k2snx\" (UID: \"162f43aa-9c5c-4148-8f41-4c1f43c9d7a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx"
Jul 17 04:57:44.379371 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:44.379248    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/162f43aa-9c5c-4148-8f41-4c1f43c9d7a9-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-k2snx\" (UID: \"162f43aa-9c5c-4148-8f41-4c1f43c9d7a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx"
Jul 17 04:57:44.379371 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:44.379278    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/162f43aa-9c5c-4148-8f41-4c1f43c9d7a9-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-k2snx\" (UID: \"162f43aa-9c5c-4148-8f41-4c1f43c9d7a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx"
Jul 17 04:57:44.379371 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:44.379302    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/162f43aa-9c5c-4148-8f41-4c1f43c9d7a9-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-k2snx\" (UID: \"162f43aa-9c5c-4148-8f41-4c1f43c9d7a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx"
Jul 17 04:57:44.379371 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:44.379327    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/162f43aa-9c5c-4148-8f41-4c1f43c9d7a9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-k2snx\" (UID: \"162f43aa-9c5c-4148-8f41-4c1f43c9d7a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx"
Jul 17 04:57:44.379371 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:44.379364    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qx9bk\" (UniqueName: \"kubernetes.io/projected/162f43aa-9c5c-4148-8f41-4c1f43c9d7a9-kube-api-access-qx9bk\") pod \"maas-default-gateway-openshift-default-5896d4dd49-k2snx\" (UID: \"162f43aa-9c5c-4148-8f41-4c1f43c9d7a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx"
Jul 17 04:57:44.379652 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:44.379401    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/162f43aa-9c5c-4148-8f41-4c1f43c9d7a9-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-k2snx\" (UID: \"162f43aa-9c5c-4148-8f41-4c1f43c9d7a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx"
Jul 17 04:57:44.379652 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:44.379465    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/162f43aa-9c5c-4148-8f41-4c1f43c9d7a9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-k2snx\" (UID: \"162f43aa-9c5c-4148-8f41-4c1f43c9d7a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx"
Jul 17 04:57:44.379753 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:44.379710    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/162f43aa-9c5c-4148-8f41-4c1f43c9d7a9-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-k2snx\" (UID: \"162f43aa-9c5c-4148-8f41-4c1f43c9d7a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx"
Jul 17 04:57:44.379851 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:44.379829    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/162f43aa-9c5c-4148-8f41-4c1f43c9d7a9-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-k2snx\" (UID: \"162f43aa-9c5c-4148-8f41-4c1f43c9d7a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx"
Jul 17 04:57:44.379916 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:44.379867    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/162f43aa-9c5c-4148-8f41-4c1f43c9d7a9-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-k2snx\" (UID: \"162f43aa-9c5c-4148-8f41-4c1f43c9d7a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx"
Jul 17 04:57:44.379916 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:44.379882    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/162f43aa-9c5c-4148-8f41-4c1f43c9d7a9-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-k2snx\" (UID: \"162f43aa-9c5c-4148-8f41-4c1f43c9d7a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx"
Jul 17 04:57:44.380283 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:44.380264    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/162f43aa-9c5c-4148-8f41-4c1f43c9d7a9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-k2snx\" (UID: \"162f43aa-9c5c-4148-8f41-4c1f43c9d7a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx"
Jul 17 04:57:44.381894 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:44.381875    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/162f43aa-9c5c-4148-8f41-4c1f43c9d7a9-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-k2snx\" (UID: \"162f43aa-9c5c-4148-8f41-4c1f43c9d7a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx"
Jul 17 04:57:44.382319 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:44.382301    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/162f43aa-9c5c-4148-8f41-4c1f43c9d7a9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-k2snx\" (UID: \"162f43aa-9c5c-4148-8f41-4c1f43c9d7a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx"
Jul 17 04:57:44.388035 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:44.388014    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/162f43aa-9c5c-4148-8f41-4c1f43c9d7a9-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-k2snx\" (UID: \"162f43aa-9c5c-4148-8f41-4c1f43c9d7a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx"
Jul 17 04:57:44.388218 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:44.388201    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qx9bk\" (UniqueName: \"kubernetes.io/projected/162f43aa-9c5c-4148-8f41-4c1f43c9d7a9-kube-api-access-qx9bk\") pod \"maas-default-gateway-openshift-default-5896d4dd49-k2snx\" (UID: \"162f43aa-9c5c-4148-8f41-4c1f43c9d7a9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx"
Jul 17 04:57:44.551288 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:44.551207    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx"
Jul 17 04:57:44.692613 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:44.687503    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx"]
Jul 17 04:57:45.024173 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:45.024137    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx" event={"ID":"162f43aa-9c5c-4148-8f41-4c1f43c9d7a9","Type":"ContainerStarted","Data":"30ad1ecb0d8f0a740a3ecb7e742c3484724bfea139c7dc6ca89c62d0fed81221"}
Jul 17 04:57:47.321561 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:47.321520    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 04:57:47.321937 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:47.321612    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 04:57:47.321937 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:47.321658    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 04:57:48.040063 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:48.040026    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx" event={"ID":"162f43aa-9c5c-4148-8f41-4c1f43c9d7a9","Type":"ContainerStarted","Data":"6c20ebaa140cf0b7ca615a11722055da4865b87ae9d59addae0277928bbc308a"}
Jul 17 04:57:48.062016 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:48.061944    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx" podStartSLOduration=1.435286741 podStartE2EDuration="4.061927295s" totalImagesPullingTime="2.626640554s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 04:57:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 04:57:48.059871786 +0000 UTC m=+455.743465457" watchObservedRunningTime="2026-07-17 04:57:48.061927295 +0000 UTC m=+455.745520952"
Jul 17 04:57:48.551588 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:48.551555    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx"
Jul 17 04:57:48.556450 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:48.556423    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx"
Jul 17 04:57:49.045135 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:49.045102    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx"
Jul 17 04:57:49.046091 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:49.046075    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx"
Jul 17 04:57:59.639842 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:59.639806    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-76fx9"]
Jul 17 04:57:59.643415 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:59.643398    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-76fx9"
Jul 17 04:57:59.645895 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:59.645861    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-h9bvw\""
Jul 17 04:57:59.645895 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:59.645875    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 17 04:57:59.645895 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:59.645862    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 17 04:57:59.646998 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:59.646957    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 17 04:57:59.650921 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:59.650885    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-76fx9"]
Jul 17 04:57:59.718897 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:59.718857    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2532e241-7b17-4e41-8688-48f03510291c-config-file\") pod \"limitador-limitador-64c94fd64f-76fx9\" (UID: \"2532e241-7b17-4e41-8688-48f03510291c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-76fx9"
Jul 17 04:57:59.719104 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:59.718931    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22wnm\" (UniqueName: \"kubernetes.io/projected/2532e241-7b17-4e41-8688-48f03510291c-kube-api-access-22wnm\") pod \"limitador-limitador-64c94fd64f-76fx9\" (UID: \"2532e241-7b17-4e41-8688-48f03510291c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-76fx9"
Jul 17 04:57:59.739373 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:59.739339    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-76fx9"]
Jul 17 04:57:59.820305 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:59.820266    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2532e241-7b17-4e41-8688-48f03510291c-config-file\") pod \"limitador-limitador-64c94fd64f-76fx9\" (UID: \"2532e241-7b17-4e41-8688-48f03510291c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-76fx9"
Jul 17 04:57:59.820485 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:59.820317    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22wnm\" (UniqueName: \"kubernetes.io/projected/2532e241-7b17-4e41-8688-48f03510291c-kube-api-access-22wnm\") pod \"limitador-limitador-64c94fd64f-76fx9\" (UID: \"2532e241-7b17-4e41-8688-48f03510291c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-76fx9"
Jul 17 04:57:59.821148 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:59.821119    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2532e241-7b17-4e41-8688-48f03510291c-config-file\") pod \"limitador-limitador-64c94fd64f-76fx9\" (UID: \"2532e241-7b17-4e41-8688-48f03510291c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-76fx9"
Jul 17 04:57:59.828898 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:59.828875    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22wnm\" (UniqueName: \"kubernetes.io/projected/2532e241-7b17-4e41-8688-48f03510291c-kube-api-access-22wnm\") pod \"limitador-limitador-64c94fd64f-76fx9\" (UID: \"2532e241-7b17-4e41-8688-48f03510291c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-76fx9"
Jul 17 04:57:59.954186 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:57:59.954098    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-76fx9"
Jul 17 04:58:00.091863 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:00.091805    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-76fx9"]
Jul 17 04:58:00.095232 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:58:00.095198    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2532e241_7b17_4e41_8688_48f03510291c.slice/crio-829ee8f81434e141e003d6fe13785fa73098766a4b5bfd0f87ea9f4e1f2cf6e5 WatchSource:0}: Error finding container 829ee8f81434e141e003d6fe13785fa73098766a4b5bfd0f87ea9f4e1f2cf6e5: Status 404 returned error can't find the container with id 829ee8f81434e141e003d6fe13785fa73098766a4b5bfd0f87ea9f4e1f2cf6e5
Jul 17 04:58:01.104446 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:01.104366    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-76fx9" event={"ID":"2532e241-7b17-4e41-8688-48f03510291c","Type":"ContainerStarted","Data":"829ee8f81434e141e003d6fe13785fa73098766a4b5bfd0f87ea9f4e1f2cf6e5"}
Jul 17 04:58:03.117069 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:03.117025    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-76fx9" event={"ID":"2532e241-7b17-4e41-8688-48f03510291c","Type":"ContainerStarted","Data":"6d1b30bedc7362adec052099622e7a220a05b1aa2b1f619ba04ae1438f5f1eae"}
Jul 17 04:58:03.117400 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:03.117145    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-76fx9"
Jul 17 04:58:03.134599 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:03.134555    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-76fx9" podStartSLOduration=1.268204259 podStartE2EDuration="4.134541233s" totalImagesPullingTime="2.866336974s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 04:57:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 04:58:03.132223599 +0000 UTC m=+470.815817276" watchObservedRunningTime="2026-07-17 04:58:03.134541233 +0000 UTC m=+470.818134890"
Jul 17 04:58:14.122701 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:14.122671    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-76fx9"
Jul 17 04:58:14.671554 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:14.671516    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-76fx9"]
Jul 17 04:58:14.671759 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:14.671730    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-76fx9" podUID="2532e241-7b17-4e41-8688-48f03510291c" containerName="limitador" containerID="cri-o://6d1b30bedc7362adec052099622e7a220a05b1aa2b1f619ba04ae1438f5f1eae" gracePeriod=30
Jul 17 04:58:15.171331 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:15.171293    2562 generic.go:356] "Generic (PLEG): container finished" podID="2532e241-7b17-4e41-8688-48f03510291c" containerID="6d1b30bedc7362adec052099622e7a220a05b1aa2b1f619ba04ae1438f5f1eae" exitCode=0
Jul 17 04:58:15.171695 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:15.171343    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-76fx9" event={"ID":"2532e241-7b17-4e41-8688-48f03510291c","Type":"ContainerDied","Data":"6d1b30bedc7362adec052099622e7a220a05b1aa2b1f619ba04ae1438f5f1eae"}
Jul 17 04:58:15.627914 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:15.627887    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-76fx9"
Jul 17 04:58:15.769575 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:15.769496    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-22wnm\" (UniqueName: \"kubernetes.io/projected/2532e241-7b17-4e41-8688-48f03510291c-kube-api-access-22wnm\") pod \"2532e241-7b17-4e41-8688-48f03510291c\" (UID: \"2532e241-7b17-4e41-8688-48f03510291c\") "
Jul 17 04:58:15.769575 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:15.769564    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2532e241-7b17-4e41-8688-48f03510291c-config-file\") pod \"2532e241-7b17-4e41-8688-48f03510291c\" (UID: \"2532e241-7b17-4e41-8688-48f03510291c\") "
Jul 17 04:58:15.769842 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:15.769818    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2532e241-7b17-4e41-8688-48f03510291c-config-file" (OuterVolumeSpecName: "config-file") pod "2532e241-7b17-4e41-8688-48f03510291c" (UID: "2532e241-7b17-4e41-8688-48f03510291c"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 04:58:15.771757 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:15.771735    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2532e241-7b17-4e41-8688-48f03510291c-kube-api-access-22wnm" (OuterVolumeSpecName: "kube-api-access-22wnm") pod "2532e241-7b17-4e41-8688-48f03510291c" (UID: "2532e241-7b17-4e41-8688-48f03510291c"). InnerVolumeSpecName "kube-api-access-22wnm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 04:58:15.870206 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:15.870181    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-22wnm\" (UniqueName: \"kubernetes.io/projected/2532e241-7b17-4e41-8688-48f03510291c-kube-api-access-22wnm\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 04:58:15.870206 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:15.870203    2562 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2532e241-7b17-4e41-8688-48f03510291c-config-file\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 04:58:16.176756 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:16.176726    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-76fx9"
Jul 17 04:58:16.177159 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:16.176724    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-76fx9" event={"ID":"2532e241-7b17-4e41-8688-48f03510291c","Type":"ContainerDied","Data":"829ee8f81434e141e003d6fe13785fa73098766a4b5bfd0f87ea9f4e1f2cf6e5"}
Jul 17 04:58:16.177159 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:16.176858    2562 scope.go:117] "RemoveContainer" containerID="6d1b30bedc7362adec052099622e7a220a05b1aa2b1f619ba04ae1438f5f1eae"
Jul 17 04:58:16.197383 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:16.197346    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-76fx9"]
Jul 17 04:58:16.199743 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:16.199723    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-76fx9"]
Jul 17 04:58:16.883725 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:16.883693    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2532e241-7b17-4e41-8688-48f03510291c" path="/var/lib/kubelet/pods/2532e241-7b17-4e41-8688-48f03510291c/volumes"
Jul 17 04:58:20.918277 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:20.918243    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-zbx4j"]
Jul 17 04:58:20.918853 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:20.918833    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2532e241-7b17-4e41-8688-48f03510291c" containerName="limitador"
Jul 17 04:58:20.918927 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:20.918859    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2532e241-7b17-4e41-8688-48f03510291c" containerName="limitador"
Jul 17 04:58:20.918996 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:20.918947    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2532e241-7b17-4e41-8688-48f03510291c" containerName="limitador"
Jul 17 04:58:20.924230 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:20.924208    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zbx4j"
Jul 17 04:58:20.926816 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:20.926795    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 17 04:58:20.926953 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:20.926847    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 17 04:58:20.926953 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:20.926914    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 04:58:20.929159 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:20.927907    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-qjm8q\""
Jul 17 04:58:20.930551 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:20.930531    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-zbx4j"]
Jul 17 04:58:21.014582 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:21.014544    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkhhn\" (UniqueName: \"kubernetes.io/projected/fd4f9eb2-a49c-46c6-909e-64392cc37e39-kube-api-access-lkhhn\") pod \"postgres-69fb4594cf-zbx4j\" (UID: \"fd4f9eb2-a49c-46c6-909e-64392cc37e39\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-zbx4j"
Jul 17 04:58:21.014754 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:21.014595    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/fd4f9eb2-a49c-46c6-909e-64392cc37e39-data\") pod \"postgres-69fb4594cf-zbx4j\" (UID: \"fd4f9eb2-a49c-46c6-909e-64392cc37e39\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-zbx4j"
Jul 17 04:58:21.115324 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:21.115269    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkhhn\" (UniqueName: \"kubernetes.io/projected/fd4f9eb2-a49c-46c6-909e-64392cc37e39-kube-api-access-lkhhn\") pod \"postgres-69fb4594cf-zbx4j\" (UID: \"fd4f9eb2-a49c-46c6-909e-64392cc37e39\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-zbx4j"
Jul 17 04:58:21.115324 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:21.115335    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/fd4f9eb2-a49c-46c6-909e-64392cc37e39-data\") pod \"postgres-69fb4594cf-zbx4j\" (UID: \"fd4f9eb2-a49c-46c6-909e-64392cc37e39\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-zbx4j"
Jul 17 04:58:21.115685 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:21.115650    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/fd4f9eb2-a49c-46c6-909e-64392cc37e39-data\") pod \"postgres-69fb4594cf-zbx4j\" (UID: \"fd4f9eb2-a49c-46c6-909e-64392cc37e39\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-zbx4j"
Jul 17 04:58:21.127394 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:21.127361    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkhhn\" (UniqueName: \"kubernetes.io/projected/fd4f9eb2-a49c-46c6-909e-64392cc37e39-kube-api-access-lkhhn\") pod \"postgres-69fb4594cf-zbx4j\" (UID: \"fd4f9eb2-a49c-46c6-909e-64392cc37e39\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-zbx4j"
Jul 17 04:58:21.236662 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:21.236570    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zbx4j"
Jul 17 04:58:21.365744 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:21.365713    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-zbx4j"]
Jul 17 04:58:21.369325 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:58:21.369299    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd4f9eb2_a49c_46c6_909e_64392cc37e39.slice/crio-a306820aa787b41f9eca6582cb1dac9ad13d497f831692a6ba460294a0446d0b WatchSource:0}: Error finding container a306820aa787b41f9eca6582cb1dac9ad13d497f831692a6ba460294a0446d0b: Status 404 returned error can't find the container with id a306820aa787b41f9eca6582cb1dac9ad13d497f831692a6ba460294a0446d0b
Jul 17 04:58:22.206273 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:22.206232    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zbx4j" event={"ID":"fd4f9eb2-a49c-46c6-909e-64392cc37e39","Type":"ContainerStarted","Data":"a306820aa787b41f9eca6582cb1dac9ad13d497f831692a6ba460294a0446d0b"}
Jul 17 04:58:27.233741 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:27.233701    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zbx4j" event={"ID":"fd4f9eb2-a49c-46c6-909e-64392cc37e39","Type":"ContainerStarted","Data":"461725c9ed28617368eddfa76c84dddfd7accdb78ba668a796b44a4c50396863"}
Jul 17 04:58:27.234161 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:27.233806    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zbx4j"
Jul 17 04:58:27.250668 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:27.250622    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zbx4j" podStartSLOduration=2.320336084 podStartE2EDuration="7.250608066s" totalImagesPullingTime="4.930271982s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 04:58:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 04:58:27.249983937 +0000 UTC m=+494.933577614" watchObservedRunningTime="2026-07-17 04:58:27.250608066 +0000 UTC m=+494.934201723"
Jul 17 04:58:33.269492 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:33.269462    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zbx4j"
Jul 17 04:58:40.279252 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:40.279214    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-75646995f5-gpxm5"]
Jul 17 04:58:40.284183 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:40.284167    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75646995f5-gpxm5"
Jul 17 04:58:40.287784 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:40.287737    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 17 04:58:40.287923 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:40.287752    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-5j47z\""
Jul 17 04:58:40.288151 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:40.288130    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 17 04:58:40.294249 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:40.294223    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75646995f5-gpxm5"]
Jul 17 04:58:40.377025 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:40.376993    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/761d2f23-c420-461a-a3d9-0568f4b30faa-webhook-cert\") pod \"maas-controller-75646995f5-gpxm5\" (UID: \"761d2f23-c420-461a-a3d9-0568f4b30faa\") " pod="opendatahub/maas-controller-75646995f5-gpxm5"
Jul 17 04:58:40.377198 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:40.377036    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vs6v\" (UniqueName: \"kubernetes.io/projected/761d2f23-c420-461a-a3d9-0568f4b30faa-kube-api-access-6vs6v\") pod \"maas-controller-75646995f5-gpxm5\" (UID: \"761d2f23-c420-461a-a3d9-0568f4b30faa\") " pod="opendatahub/maas-controller-75646995f5-gpxm5"
Jul 17 04:58:40.443636 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:40.443604    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-859ff64978-lzph8"]
Jul 17 04:58:40.445997 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:40.445957    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-859ff64978-lzph8"
Jul 17 04:58:40.455322 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:40.455294    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-859ff64978-lzph8"]
Jul 17 04:58:40.478236 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:40.478210    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/761d2f23-c420-461a-a3d9-0568f4b30faa-webhook-cert\") pod \"maas-controller-75646995f5-gpxm5\" (UID: \"761d2f23-c420-461a-a3d9-0568f4b30faa\") " pod="opendatahub/maas-controller-75646995f5-gpxm5"
Jul 17 04:58:40.478387 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:40.478246    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vs6v\" (UniqueName: \"kubernetes.io/projected/761d2f23-c420-461a-a3d9-0568f4b30faa-kube-api-access-6vs6v\") pod \"maas-controller-75646995f5-gpxm5\" (UID: \"761d2f23-c420-461a-a3d9-0568f4b30faa\") " pod="opendatahub/maas-controller-75646995f5-gpxm5"
Jul 17 04:58:40.478387 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:40.478277    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9d66x\" (UniqueName: \"kubernetes.io/projected/b63a7320-d527-4f4c-8d12-ad1852089444-kube-api-access-9d66x\") pod \"maas-controller-859ff64978-lzph8\" (UID: \"b63a7320-d527-4f4c-8d12-ad1852089444\") " pod="opendatahub/maas-controller-859ff64978-lzph8"
Jul 17 04:58:40.478387 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:40.478308    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b63a7320-d527-4f4c-8d12-ad1852089444-webhook-cert\") pod \"maas-controller-859ff64978-lzph8\" (UID: \"b63a7320-d527-4f4c-8d12-ad1852089444\") " pod="opendatahub/maas-controller-859ff64978-lzph8"
Jul 17 04:58:40.480701 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:40.480683    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/761d2f23-c420-461a-a3d9-0568f4b30faa-webhook-cert\") pod \"maas-controller-75646995f5-gpxm5\" (UID: \"761d2f23-c420-461a-a3d9-0568f4b30faa\") " pod="opendatahub/maas-controller-75646995f5-gpxm5"
Jul 17 04:58:40.486455 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:40.486436    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vs6v\" (UniqueName: \"kubernetes.io/projected/761d2f23-c420-461a-a3d9-0568f4b30faa-kube-api-access-6vs6v\") pod \"maas-controller-75646995f5-gpxm5\" (UID: \"761d2f23-c420-461a-a3d9-0568f4b30faa\") " pod="opendatahub/maas-controller-75646995f5-gpxm5"
Jul 17 04:58:40.578737 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:40.578705    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9d66x\" (UniqueName: \"kubernetes.io/projected/b63a7320-d527-4f4c-8d12-ad1852089444-kube-api-access-9d66x\") pod \"maas-controller-859ff64978-lzph8\" (UID: \"b63a7320-d527-4f4c-8d12-ad1852089444\") " pod="opendatahub/maas-controller-859ff64978-lzph8"
Jul 17 04:58:40.578928 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:40.578758    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b63a7320-d527-4f4c-8d12-ad1852089444-webhook-cert\") pod \"maas-controller-859ff64978-lzph8\" (UID: \"b63a7320-d527-4f4c-8d12-ad1852089444\") " pod="opendatahub/maas-controller-859ff64978-lzph8"
Jul 17 04:58:40.581274 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:40.581255    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b63a7320-d527-4f4c-8d12-ad1852089444-webhook-cert\") pod \"maas-controller-859ff64978-lzph8\" (UID: \"b63a7320-d527-4f4c-8d12-ad1852089444\") " pod="opendatahub/maas-controller-859ff64978-lzph8"
Jul 17 04:58:40.586876 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:40.586855    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9d66x\" (UniqueName: \"kubernetes.io/projected/b63a7320-d527-4f4c-8d12-ad1852089444-kube-api-access-9d66x\") pod \"maas-controller-859ff64978-lzph8\" (UID: \"b63a7320-d527-4f4c-8d12-ad1852089444\") " pod="opendatahub/maas-controller-859ff64978-lzph8"
Jul 17 04:58:40.597720 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:40.597698    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75646995f5-gpxm5"
Jul 17 04:58:40.726301 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:40.726276    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75646995f5-gpxm5"]
Jul 17 04:58:40.728688 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:58:40.728658    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod761d2f23_c420_461a_a3d9_0568f4b30faa.slice/crio-58b2318a1187b5787577fc938c12fbef9162eb4768ec8039e0e05032a7c8db83 WatchSource:0}: Error finding container 58b2318a1187b5787577fc938c12fbef9162eb4768ec8039e0e05032a7c8db83: Status 404 returned error can't find the container with id 58b2318a1187b5787577fc938c12fbef9162eb4768ec8039e0e05032a7c8db83
Jul 17 04:58:40.757814 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:40.757788    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-859ff64978-lzph8"
Jul 17 04:58:40.885831 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:40.885808    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-859ff64978-lzph8"]
Jul 17 04:58:40.887782 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:58:40.887759    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb63a7320_d527_4f4c_8d12_ad1852089444.slice/crio-f31b27c5e7dad40ad7c246e6cda90c838a5ac59d8b04ea732ffd4f38cb7f7f34 WatchSource:0}: Error finding container f31b27c5e7dad40ad7c246e6cda90c838a5ac59d8b04ea732ffd4f38cb7f7f34: Status 404 returned error can't find the container with id f31b27c5e7dad40ad7c246e6cda90c838a5ac59d8b04ea732ffd4f38cb7f7f34
Jul 17 04:58:41.303302 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:41.303208    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75646995f5-gpxm5" event={"ID":"761d2f23-c420-461a-a3d9-0568f4b30faa","Type":"ContainerStarted","Data":"58b2318a1187b5787577fc938c12fbef9162eb4768ec8039e0e05032a7c8db83"}
Jul 17 04:58:41.304402 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:41.304377    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-859ff64978-lzph8" event={"ID":"b63a7320-d527-4f4c-8d12-ad1852089444","Type":"ContainerStarted","Data":"f31b27c5e7dad40ad7c246e6cda90c838a5ac59d8b04ea732ffd4f38cb7f7f34"}
Jul 17 04:58:44.323829 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:44.323777    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-859ff64978-lzph8" event={"ID":"b63a7320-d527-4f4c-8d12-ad1852089444","Type":"ContainerStarted","Data":"add479b236792062b096b650a52c6996a0c962693ecb1e67dc172279dc2ddca3"}
Jul 17 04:58:44.324372 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:44.323870    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-859ff64978-lzph8"
Jul 17 04:58:44.325284 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:44.325262    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75646995f5-gpxm5" event={"ID":"761d2f23-c420-461a-a3d9-0568f4b30faa","Type":"ContainerStarted","Data":"df2f04f3bb71f7a775a4e193f53cc674571bcc65ad807d9aaee8abffb0e281a2"}
Jul 17 04:58:44.325391 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:44.325376    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-75646995f5-gpxm5"
Jul 17 04:58:44.342548 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:44.342492    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-859ff64978-lzph8" podStartSLOduration=1.64650331 podStartE2EDuration="4.342476575s" totalImagesPullingTime="2.695973265s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 04:58:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 04:58:44.342410967 +0000 UTC m=+512.026004646" watchObservedRunningTime="2026-07-17 04:58:44.342476575 +0000 UTC m=+512.026070235"
Jul 17 04:58:44.360515 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:44.360479    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-75646995f5-gpxm5" podStartSLOduration=1.516497684 podStartE2EDuration="4.360468232s" totalImagesPullingTime="2.843970548s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 04:58:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 04:58:44.358749712 +0000 UTC m=+512.042343392" watchObservedRunningTime="2026-07-17 04:58:44.360468232 +0000 UTC m=+512.044061890"
Jul 17 04:58:55.332700 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:55.332669    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-75646995f5-gpxm5"
Jul 17 04:58:55.333171 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:55.332742    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-859ff64978-lzph8"
Jul 17 04:58:55.395261 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:55.395229    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-75646995f5-gpxm5"]
Jul 17 04:58:55.395506 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:55.395456    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-75646995f5-gpxm5" podUID="761d2f23-c420-461a-a3d9-0568f4b30faa" containerName="manager" containerID="cri-o://df2f04f3bb71f7a775a4e193f53cc674571bcc65ad807d9aaee8abffb0e281a2" gracePeriod=30
Jul 17 04:58:55.635657 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:55.635627    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75646995f5-gpxm5"
Jul 17 04:58:55.711521 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:55.711487    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vs6v\" (UniqueName: \"kubernetes.io/projected/761d2f23-c420-461a-a3d9-0568f4b30faa-kube-api-access-6vs6v\") pod \"761d2f23-c420-461a-a3d9-0568f4b30faa\" (UID: \"761d2f23-c420-461a-a3d9-0568f4b30faa\") "
Jul 17 04:58:55.711698 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:55.711556    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/761d2f23-c420-461a-a3d9-0568f4b30faa-webhook-cert\") pod \"761d2f23-c420-461a-a3d9-0568f4b30faa\" (UID: \"761d2f23-c420-461a-a3d9-0568f4b30faa\") "
Jul 17 04:58:55.713714 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:55.713688    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/761d2f23-c420-461a-a3d9-0568f4b30faa-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "761d2f23-c420-461a-a3d9-0568f4b30faa" (UID: "761d2f23-c420-461a-a3d9-0568f4b30faa"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 04:58:55.713817 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:55.713756    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/761d2f23-c420-461a-a3d9-0568f4b30faa-kube-api-access-6vs6v" (OuterVolumeSpecName: "kube-api-access-6vs6v") pod "761d2f23-c420-461a-a3d9-0568f4b30faa" (UID: "761d2f23-c420-461a-a3d9-0568f4b30faa"). InnerVolumeSpecName "kube-api-access-6vs6v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 04:58:55.813072 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:55.813042    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vs6v\" (UniqueName: \"kubernetes.io/projected/761d2f23-c420-461a-a3d9-0568f4b30faa-kube-api-access-6vs6v\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 04:58:55.813072 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:55.813065    2562 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/761d2f23-c420-461a-a3d9-0568f4b30faa-webhook-cert\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 04:58:56.381269 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:56.381237    2562 generic.go:356] "Generic (PLEG): container finished" podID="761d2f23-c420-461a-a3d9-0568f4b30faa" containerID="df2f04f3bb71f7a775a4e193f53cc674571bcc65ad807d9aaee8abffb0e281a2" exitCode=0
Jul 17 04:58:56.381729 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:56.381284    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75646995f5-gpxm5"
Jul 17 04:58:56.381729 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:56.381323    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75646995f5-gpxm5" event={"ID":"761d2f23-c420-461a-a3d9-0568f4b30faa","Type":"ContainerDied","Data":"df2f04f3bb71f7a775a4e193f53cc674571bcc65ad807d9aaee8abffb0e281a2"}
Jul 17 04:58:56.381729 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:56.381377    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75646995f5-gpxm5" event={"ID":"761d2f23-c420-461a-a3d9-0568f4b30faa","Type":"ContainerDied","Data":"58b2318a1187b5787577fc938c12fbef9162eb4768ec8039e0e05032a7c8db83"}
Jul 17 04:58:56.381729 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:56.381400    2562 scope.go:117] "RemoveContainer" containerID="df2f04f3bb71f7a775a4e193f53cc674571bcc65ad807d9aaee8abffb0e281a2"
Jul 17 04:58:56.390286 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:56.390271    2562 scope.go:117] "RemoveContainer" containerID="df2f04f3bb71f7a775a4e193f53cc674571bcc65ad807d9aaee8abffb0e281a2"
Jul 17 04:58:56.390546 ip-10-0-132-19 kubenswrapper[2562]: E0717 04:58:56.390529    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df2f04f3bb71f7a775a4e193f53cc674571bcc65ad807d9aaee8abffb0e281a2\": container with ID starting with df2f04f3bb71f7a775a4e193f53cc674571bcc65ad807d9aaee8abffb0e281a2 not found: ID does not exist" containerID="df2f04f3bb71f7a775a4e193f53cc674571bcc65ad807d9aaee8abffb0e281a2"
Jul 17 04:58:56.390608 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:56.390559    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df2f04f3bb71f7a775a4e193f53cc674571bcc65ad807d9aaee8abffb0e281a2"} err="failed to get container status \"df2f04f3bb71f7a775a4e193f53cc674571bcc65ad807d9aaee8abffb0e281a2\": rpc error: code = NotFound desc = could not find container \"df2f04f3bb71f7a775a4e193f53cc674571bcc65ad807d9aaee8abffb0e281a2\": container with ID starting with df2f04f3bb71f7a775a4e193f53cc674571bcc65ad807d9aaee8abffb0e281a2 not found: ID does not exist"
Jul 17 04:58:56.401065 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:56.401041    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-75646995f5-gpxm5"]
Jul 17 04:58:56.404337 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:56.404315    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-75646995f5-gpxm5"]
Jul 17 04:58:56.883370 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:58:56.883335    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="761d2f23-c420-461a-a3d9-0568f4b30faa" path="/var/lib/kubelet/pods/761d2f23-c420-461a-a3d9-0568f4b30faa/volumes"
Jul 17 04:59:17.286114 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:17.286081    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7954f7d7ff-j7k45"]
Jul 17 04:59:17.286562 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:17.286458    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="761d2f23-c420-461a-a3d9-0568f4b30faa" containerName="manager"
Jul 17 04:59:17.286562 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:17.286470    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="761d2f23-c420-461a-a3d9-0568f4b30faa" containerName="manager"
Jul 17 04:59:17.286562 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:17.286520    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="761d2f23-c420-461a-a3d9-0568f4b30faa" containerName="manager"
Jul 17 04:59:17.293044 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:17.293020    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-j7k45"
Jul 17 04:59:17.297181 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:17.297156    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 17 04:59:17.311034 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:17.311007    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7954f7d7ff-j7k45"]
Jul 17 04:59:17.392945 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:17.392910    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l82mz\" (UniqueName: \"kubernetes.io/projected/8c259023-8aaa-48ef-b9b2-6818d76290eb-kube-api-access-l82mz\") pod \"maas-api-7954f7d7ff-j7k45\" (UID: \"8c259023-8aaa-48ef-b9b2-6818d76290eb\") " pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-j7k45"
Jul 17 04:59:17.393143 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:17.393030    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c259023-8aaa-48ef-b9b2-6818d76290eb-maas-api-tls\") pod \"maas-api-7954f7d7ff-j7k45\" (UID: \"8c259023-8aaa-48ef-b9b2-6818d76290eb\") " pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-j7k45"
Jul 17 04:59:17.393143 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:17.393084    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c259023-8aaa-48ef-b9b2-6818d76290eb-openshift-service-ca\") pod \"maas-api-7954f7d7ff-j7k45\" (UID: \"8c259023-8aaa-48ef-b9b2-6818d76290eb\") " pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-j7k45"
Jul 17 04:59:17.494173 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:17.494141    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l82mz\" (UniqueName: \"kubernetes.io/projected/8c259023-8aaa-48ef-b9b2-6818d76290eb-kube-api-access-l82mz\") pod \"maas-api-7954f7d7ff-j7k45\" (UID: \"8c259023-8aaa-48ef-b9b2-6818d76290eb\") " pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-j7k45"
Jul 17 04:59:17.494360 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:17.494291    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c259023-8aaa-48ef-b9b2-6818d76290eb-maas-api-tls\") pod \"maas-api-7954f7d7ff-j7k45\" (UID: \"8c259023-8aaa-48ef-b9b2-6818d76290eb\") " pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-j7k45"
Jul 17 04:59:17.494360 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:17.494337    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c259023-8aaa-48ef-b9b2-6818d76290eb-openshift-service-ca\") pod \"maas-api-7954f7d7ff-j7k45\" (UID: \"8c259023-8aaa-48ef-b9b2-6818d76290eb\") " pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-j7k45"
Jul 17 04:59:17.495035 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:17.495009    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c259023-8aaa-48ef-b9b2-6818d76290eb-openshift-service-ca\") pod \"maas-api-7954f7d7ff-j7k45\" (UID: \"8c259023-8aaa-48ef-b9b2-6818d76290eb\") " pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-j7k45"
Jul 17 04:59:17.497253 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:17.497226    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c259023-8aaa-48ef-b9b2-6818d76290eb-maas-api-tls\") pod \"maas-api-7954f7d7ff-j7k45\" (UID: \"8c259023-8aaa-48ef-b9b2-6818d76290eb\") " pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-j7k45"
Jul 17 04:59:17.504744 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:17.504713    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l82mz\" (UniqueName: \"kubernetes.io/projected/8c259023-8aaa-48ef-b9b2-6818d76290eb-kube-api-access-l82mz\") pod \"maas-api-7954f7d7ff-j7k45\" (UID: \"8c259023-8aaa-48ef-b9b2-6818d76290eb\") " pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-j7k45"
Jul 17 04:59:17.603870 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:17.603638    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-j7k45"
Jul 17 04:59:17.747013 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:17.746980    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7954f7d7ff-j7k45"]
Jul 17 04:59:17.748842 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:59:17.748818    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c259023_8aaa_48ef_b9b2_6818d76290eb.slice/crio-f00430213ba5c1d3b83890b517d3265e12f95f9a6bf5950b27296f4bdf655d85 WatchSource:0}: Error finding container f00430213ba5c1d3b83890b517d3265e12f95f9a6bf5950b27296f4bdf655d85: Status 404 returned error can't find the container with id f00430213ba5c1d3b83890b517d3265e12f95f9a6bf5950b27296f4bdf655d85
Jul 17 04:59:17.896584 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:17.896509    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-qc8sv"]
Jul 17 04:59:17.901185 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:17.901167    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-qc8sv"
Jul 17 04:59:17.903571 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:17.903552    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 17 04:59:17.903677 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:17.903589    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-ffp7c\""
Jul 17 04:59:17.910616 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:17.910597    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-qc8sv"]
Jul 17 04:59:18.001248 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:18.001201    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc5f1ecb-28e8-49ce-89d7-b841f10e04b4-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-qc8sv\" (UID: \"fc5f1ecb-28e8-49ce-89d7-b841f10e04b4\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-qc8sv"
Jul 17 04:59:18.001985 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:18.001463    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdg8r\" (UniqueName: \"kubernetes.io/projected/fc5f1ecb-28e8-49ce-89d7-b841f10e04b4-kube-api-access-zdg8r\") pod \"payload-pre-processing-7677b774bb-qc8sv\" (UID: \"fc5f1ecb-28e8-49ce-89d7-b841f10e04b4\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-qc8sv"
Jul 17 04:59:18.102164 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:18.102113    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdg8r\" (UniqueName: \"kubernetes.io/projected/fc5f1ecb-28e8-49ce-89d7-b841f10e04b4-kube-api-access-zdg8r\") pod \"payload-pre-processing-7677b774bb-qc8sv\" (UID: \"fc5f1ecb-28e8-49ce-89d7-b841f10e04b4\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-qc8sv"
Jul 17 04:59:18.102356 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:18.102194    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc5f1ecb-28e8-49ce-89d7-b841f10e04b4-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-qc8sv\" (UID: \"fc5f1ecb-28e8-49ce-89d7-b841f10e04b4\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-qc8sv"
Jul 17 04:59:18.102805 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:18.102785    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc5f1ecb-28e8-49ce-89d7-b841f10e04b4-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-qc8sv\" (UID: \"fc5f1ecb-28e8-49ce-89d7-b841f10e04b4\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-qc8sv"
Jul 17 04:59:18.111511 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:18.111487    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdg8r\" (UniqueName: \"kubernetes.io/projected/fc5f1ecb-28e8-49ce-89d7-b841f10e04b4-kube-api-access-zdg8r\") pod \"payload-pre-processing-7677b774bb-qc8sv\" (UID: \"fc5f1ecb-28e8-49ce-89d7-b841f10e04b4\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-qc8sv"
Jul 17 04:59:18.213652 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:18.213559    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-qc8sv"
Jul 17 04:59:18.485405 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:18.485315    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-j7k45" event={"ID":"8c259023-8aaa-48ef-b9b2-6818d76290eb","Type":"ContainerStarted","Data":"f00430213ba5c1d3b83890b517d3265e12f95f9a6bf5950b27296f4bdf655d85"}
Jul 17 04:59:18.563008 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:18.562979    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-qc8sv"]
Jul 17 04:59:18.565241 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:59:18.565213    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc5f1ecb_28e8_49ce_89d7_b841f10e04b4.slice/crio-1d413fbf8384e31d1f11a38b61fbfc6e7b5265ff22d8fdc57e18a003eccba708 WatchSource:0}: Error finding container 1d413fbf8384e31d1f11a38b61fbfc6e7b5265ff22d8fdc57e18a003eccba708: Status 404 returned error can't find the container with id 1d413fbf8384e31d1f11a38b61fbfc6e7b5265ff22d8fdc57e18a003eccba708
Jul 17 04:59:19.491365 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:19.491331    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-qc8sv" event={"ID":"fc5f1ecb-28e8-49ce-89d7-b841f10e04b4","Type":"ContainerStarted","Data":"1d413fbf8384e31d1f11a38b61fbfc6e7b5265ff22d8fdc57e18a003eccba708"}
Jul 17 04:59:20.497488 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:20.497452    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-j7k45" event={"ID":"8c259023-8aaa-48ef-b9b2-6818d76290eb","Type":"ContainerStarted","Data":"f0d6b0200b1deb24f5a84ccdf9fd33c7dc15a6ac31f93d6891bb3a1dc69a2a64"}
Jul 17 04:59:20.498061 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:20.498004    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-j7k45"
Jul 17 04:59:20.517872 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:20.517822    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-j7k45" podStartSLOduration=1.783240554 podStartE2EDuration="3.517807517s" totalImagesPullingTime="1.734566963s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 04:59:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 04:59:20.515639298 +0000 UTC m=+548.199232968" watchObservedRunningTime="2026-07-17 04:59:20.517807517 +0000 UTC m=+548.201401174"
Jul 17 04:59:21.504997 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:21.504944    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-qc8sv" event={"ID":"fc5f1ecb-28e8-49ce-89d7-b841f10e04b4","Type":"ContainerStarted","Data":"90829ae8d0980da294af76191e52c2b08da334febe9df88421ed0bbde8f7f747"}
Jul 17 04:59:21.521813 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:21.521764    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-qc8sv" podStartSLOduration=2.178459444 podStartE2EDuration="4.521748366s" totalImagesPullingTime="2.343288922s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 04:59:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 04:59:21.520919512 +0000 UTC m=+549.204513195" watchObservedRunningTime="2026-07-17 04:59:21.521748366 +0000 UTC m=+549.205342023"
Jul 17 04:59:22.509892 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:22.509858    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-qc8sv"
Jul 17 04:59:26.531333 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:26.531298    2562 generic.go:356] "Generic (PLEG): container finished" podID="162f43aa-9c5c-4148-8f41-4c1f43c9d7a9" containerID="6c20ebaa140cf0b7ca615a11722055da4865b87ae9d59addae0277928bbc308a" exitCode=0
Jul 17 04:59:26.531727 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:26.531351    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx" event={"ID":"162f43aa-9c5c-4148-8f41-4c1f43c9d7a9","Type":"ContainerDied","Data":"6c20ebaa140cf0b7ca615a11722055da4865b87ae9d59addae0277928bbc308a"}
Jul 17 04:59:26.531727 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:26.531678    2562 scope.go:117] "RemoveContainer" containerID="6c20ebaa140cf0b7ca615a11722055da4865b87ae9d59addae0277928bbc308a"
Jul 17 04:59:26.532605 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:26.532569    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 04:59:26.532671 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:26.532657    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 04:59:26.532710 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:26.532699    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 04:59:27.515031 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:27.515006    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-j7k45"
Jul 17 04:59:27.538822 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:27.538771    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx" event={"ID":"162f43aa-9c5c-4148-8f41-4c1f43c9d7a9","Type":"ContainerStarted","Data":"b46218dd5da18df79d4255213546848b68ea47055bef8b161d13599e5998c9e6"}
Jul 17 04:59:27.552029 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:27.551999    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx"
Jul 17 04:59:27.552283 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:27.552261    2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-k2snx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body=
Jul 17 04:59:27.552364 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:27.552313    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx" podUID="162f43aa-9c5c-4148-8f41-4c1f43c9d7a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused"
Jul 17 04:59:28.551977 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:28.551940    2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-k2snx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body=
Jul 17 04:59:28.552354 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:28.552013    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx" podUID="162f43aa-9c5c-4148-8f41-4c1f43c9d7a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused"
Jul 17 04:59:29.552620 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:29.552587    2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-k2snx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body=
Jul 17 04:59:29.553023 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:29.552639    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx" podUID="162f43aa-9c5c-4148-8f41-4c1f43c9d7a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused"
Jul 17 04:59:30.551787 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:30.551751    2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-k2snx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body=
Jul 17 04:59:30.551977 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:30.551805    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx" podUID="162f43aa-9c5c-4148-8f41-4c1f43c9d7a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused"
Jul 17 04:59:31.556112 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:31.556084    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx"
Jul 17 04:59:31.556514 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:31.556402    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx"
Jul 17 04:59:31.557076 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:31.557058    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-k2snx"
Jul 17 04:59:33.517290 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:33.517262    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-qc8sv"
Jul 17 04:59:51.142168 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:51.142132    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b"]
Jul 17 04:59:51.150185 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:51.150165    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b"
Jul 17 04:59:51.153697 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:51.153671    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b"]
Jul 17 04:59:51.154482 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:51.154461    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-q7v2x\""
Jul 17 04:59:51.154585 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:51.154461    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 17 04:59:51.154649 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:51.154604    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 17 04:59:51.154649 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:51.154462    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 17 04:59:51.302837 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:51.302808    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28ddd366-d845-4f5a-984c-08e9805563a1-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b\" (UID: \"28ddd366-d845-4f5a-984c-08e9805563a1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b"
Jul 17 04:59:51.302837 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:51.302839    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28ddd366-d845-4f5a-984c-08e9805563a1-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b\" (UID: \"28ddd366-d845-4f5a-984c-08e9805563a1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b"
Jul 17 04:59:51.303130 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:51.302871    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28ddd366-d845-4f5a-984c-08e9805563a1-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b\" (UID: \"28ddd366-d845-4f5a-984c-08e9805563a1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b"
Jul 17 04:59:51.303130 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:51.302895    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5dxx\" (UniqueName: \"kubernetes.io/projected/28ddd366-d845-4f5a-984c-08e9805563a1-kube-api-access-p5dxx\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b\" (UID: \"28ddd366-d845-4f5a-984c-08e9805563a1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b"
Jul 17 04:59:51.303130 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:51.303058    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28ddd366-d845-4f5a-984c-08e9805563a1-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b\" (UID: \"28ddd366-d845-4f5a-984c-08e9805563a1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b"
Jul 17 04:59:51.303130 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:51.303097    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28ddd366-d845-4f5a-984c-08e9805563a1-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b\" (UID: \"28ddd366-d845-4f5a-984c-08e9805563a1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b"
Jul 17 04:59:51.403865 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:51.403783    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5dxx\" (UniqueName: \"kubernetes.io/projected/28ddd366-d845-4f5a-984c-08e9805563a1-kube-api-access-p5dxx\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b\" (UID: \"28ddd366-d845-4f5a-984c-08e9805563a1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b"
Jul 17 04:59:51.403865 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:51.403849    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28ddd366-d845-4f5a-984c-08e9805563a1-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b\" (UID: \"28ddd366-d845-4f5a-984c-08e9805563a1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b"
Jul 17 04:59:51.404090 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:51.403871    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28ddd366-d845-4f5a-984c-08e9805563a1-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b\" (UID: \"28ddd366-d845-4f5a-984c-08e9805563a1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b"
Jul 17 04:59:51.404090 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:51.403910    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28ddd366-d845-4f5a-984c-08e9805563a1-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b\" (UID: \"28ddd366-d845-4f5a-984c-08e9805563a1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b"
Jul 17 04:59:51.404090 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:51.403926    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28ddd366-d845-4f5a-984c-08e9805563a1-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b\" (UID: \"28ddd366-d845-4f5a-984c-08e9805563a1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b"
Jul 17 04:59:51.404227 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:51.404102    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28ddd366-d845-4f5a-984c-08e9805563a1-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b\" (UID: \"28ddd366-d845-4f5a-984c-08e9805563a1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b"
Jul 17 04:59:51.404354 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:51.404331    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28ddd366-d845-4f5a-984c-08e9805563a1-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b\" (UID: \"28ddd366-d845-4f5a-984c-08e9805563a1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b"
Jul 17 04:59:51.404419 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:51.404372    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28ddd366-d845-4f5a-984c-08e9805563a1-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b\" (UID: \"28ddd366-d845-4f5a-984c-08e9805563a1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b"
Jul 17 04:59:51.404477 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:51.404460    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28ddd366-d845-4f5a-984c-08e9805563a1-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b\" (UID: \"28ddd366-d845-4f5a-984c-08e9805563a1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b"
Jul 17 04:59:51.406339 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:51.406319    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28ddd366-d845-4f5a-984c-08e9805563a1-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b\" (UID: \"28ddd366-d845-4f5a-984c-08e9805563a1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b"
Jul 17 04:59:51.406534 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:51.406517    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28ddd366-d845-4f5a-984c-08e9805563a1-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b\" (UID: \"28ddd366-d845-4f5a-984c-08e9805563a1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b"
Jul 17 04:59:51.412857 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:51.412835    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5dxx\" (UniqueName: \"kubernetes.io/projected/28ddd366-d845-4f5a-984c-08e9805563a1-kube-api-access-p5dxx\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b\" (UID: \"28ddd366-d845-4f5a-984c-08e9805563a1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b"
Jul 17 04:59:51.461921 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:51.461898    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b"
Jul 17 04:59:51.594530 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:51.594391    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b"]
Jul 17 04:59:51.596667 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:59:51.596641    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28ddd366_d845_4f5a_984c_08e9805563a1.slice/crio-89dde4e3b921c33f98090dfeb41b9bfc7b63891ffddaf1aebdd627e93984fc1a WatchSource:0}: Error finding container 89dde4e3b921c33f98090dfeb41b9bfc7b63891ffddaf1aebdd627e93984fc1a: Status 404 returned error can't find the container with id 89dde4e3b921c33f98090dfeb41b9bfc7b63891ffddaf1aebdd627e93984fc1a
Jul 17 04:59:51.653931 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:51.653855    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b" event={"ID":"28ddd366-d845-4f5a-984c-08e9805563a1","Type":"ContainerStarted","Data":"89dde4e3b921c33f98090dfeb41b9bfc7b63891ffddaf1aebdd627e93984fc1a"}
Jul 17 04:59:53.665448 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:53.665412    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b" event={"ID":"28ddd366-d845-4f5a-984c-08e9805563a1","Type":"ContainerStarted","Data":"ba60b1ebd84c404d0265750232f365e0cd5eb1bda2b357e741e1ee1acdf0687f"}
Jul 17 04:59:53.693887 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:53.688524    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b" podStartSLOduration=1.630986326 podStartE2EDuration="2.688504804s" totalImagesPullingTime="1.057518478s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 04:59:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 04:59:53.688339962 +0000 UTC m=+581.371933620" watchObservedRunningTime="2026-07-17 04:59:53.688504804 +0000 UTC m=+581.372098462"
Jul 17 04:59:58.625977 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:58.625931    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-bjn2w"]
Jul 17 04:59:58.628560 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:58.628540    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bjn2w"
Jul 17 04:59:58.630991 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:58.630957    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 17 04:59:58.637233 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:58.637201    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-bjn2w"]
Jul 17 04:59:58.776330 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:58.776303    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e694bf8-4bfc-49c4-82e1-6b46739868cc-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bjn2w\" (UID: \"5e694bf8-4bfc-49c4-82e1-6b46739868cc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bjn2w"
Jul 17 04:59:58.776485 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:58.776354    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e694bf8-4bfc-49c4-82e1-6b46739868cc-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bjn2w\" (UID: \"5e694bf8-4bfc-49c4-82e1-6b46739868cc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bjn2w"
Jul 17 04:59:58.776485 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:58.776399    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e694bf8-4bfc-49c4-82e1-6b46739868cc-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bjn2w\" (UID: \"5e694bf8-4bfc-49c4-82e1-6b46739868cc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bjn2w"
Jul 17 04:59:58.776485 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:58.776436    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e694bf8-4bfc-49c4-82e1-6b46739868cc-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bjn2w\" (UID: \"5e694bf8-4bfc-49c4-82e1-6b46739868cc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bjn2w"
Jul 17 04:59:58.776485 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:58.776461    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gskgt\" (UniqueName: \"kubernetes.io/projected/5e694bf8-4bfc-49c4-82e1-6b46739868cc-kube-api-access-gskgt\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bjn2w\" (UID: \"5e694bf8-4bfc-49c4-82e1-6b46739868cc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bjn2w"
Jul 17 04:59:58.776643 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:58.776496    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e694bf8-4bfc-49c4-82e1-6b46739868cc-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bjn2w\" (UID: \"5e694bf8-4bfc-49c4-82e1-6b46739868cc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bjn2w"
Jul 17 04:59:58.877421 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:58.877341    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e694bf8-4bfc-49c4-82e1-6b46739868cc-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bjn2w\" (UID: \"5e694bf8-4bfc-49c4-82e1-6b46739868cc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bjn2w"
Jul 17 04:59:58.877421 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:58.877392    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e694bf8-4bfc-49c4-82e1-6b46739868cc-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bjn2w\" (UID: \"5e694bf8-4bfc-49c4-82e1-6b46739868cc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bjn2w"
Jul 17 04:59:58.877627 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:58.877569    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gskgt\" (UniqueName: \"kubernetes.io/projected/5e694bf8-4bfc-49c4-82e1-6b46739868cc-kube-api-access-gskgt\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bjn2w\" (UID: \"5e694bf8-4bfc-49c4-82e1-6b46739868cc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bjn2w"
Jul 17 04:59:58.877690 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:58.877625    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e694bf8-4bfc-49c4-82e1-6b46739868cc-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bjn2w\" (UID: \"5e694bf8-4bfc-49c4-82e1-6b46739868cc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bjn2w"
Jul 17 04:59:58.877764 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:58.877743    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e694bf8-4bfc-49c4-82e1-6b46739868cc-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bjn2w\" (UID: \"5e694bf8-4bfc-49c4-82e1-6b46739868cc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bjn2w"
Jul 17 04:59:58.877819 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:58.877787    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e694bf8-4bfc-49c4-82e1-6b46739868cc-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bjn2w\" (UID: \"5e694bf8-4bfc-49c4-82e1-6b46739868cc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bjn2w"
Jul 17 04:59:58.877878 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:58.877814    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e694bf8-4bfc-49c4-82e1-6b46739868cc-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bjn2w\" (UID: \"5e694bf8-4bfc-49c4-82e1-6b46739868cc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bjn2w"
Jul 17 04:59:58.878107 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:58.878073    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e694bf8-4bfc-49c4-82e1-6b46739868cc-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bjn2w\" (UID: \"5e694bf8-4bfc-49c4-82e1-6b46739868cc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bjn2w"
Jul 17 04:59:58.878404 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:58.878153    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e694bf8-4bfc-49c4-82e1-6b46739868cc-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bjn2w\" (UID: \"5e694bf8-4bfc-49c4-82e1-6b46739868cc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bjn2w"
Jul 17 04:59:58.879957 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:58.879931    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e694bf8-4bfc-49c4-82e1-6b46739868cc-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bjn2w\" (UID: \"5e694bf8-4bfc-49c4-82e1-6b46739868cc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bjn2w"
Jul 17 04:59:58.880372 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:58.880350    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e694bf8-4bfc-49c4-82e1-6b46739868cc-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bjn2w\" (UID: \"5e694bf8-4bfc-49c4-82e1-6b46739868cc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bjn2w"
Jul 17 04:59:58.886772 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:58.886753    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gskgt\" (UniqueName: \"kubernetes.io/projected/5e694bf8-4bfc-49c4-82e1-6b46739868cc-kube-api-access-gskgt\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bjn2w\" (UID: \"5e694bf8-4bfc-49c4-82e1-6b46739868cc\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bjn2w"
Jul 17 04:59:58.940362 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:58.940332    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bjn2w"
Jul 17 04:59:59.092771 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:59.092747    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-bjn2w"]
Jul 17 04:59:59.094302 ip-10-0-132-19 kubenswrapper[2562]: W0717 04:59:59.094275    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e694bf8_4bfc_49c4_82e1_6b46739868cc.slice/crio-81f2ed236442497679dca977e4140c12cca5b79bb24866434b4bebf0b3a56ec8 WatchSource:0}: Error finding container 81f2ed236442497679dca977e4140c12cca5b79bb24866434b4bebf0b3a56ec8: Status 404 returned error can't find the container with id 81f2ed236442497679dca977e4140c12cca5b79bb24866434b4bebf0b3a56ec8
Jul 17 04:59:59.694177 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:59.694137    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bjn2w" event={"ID":"5e694bf8-4bfc-49c4-82e1-6b46739868cc","Type":"ContainerStarted","Data":"6db8ee5657bb1e7745ec00299ef7400d4b31bf360282f9f7e57b0d291cd67d5d"}
Jul 17 04:59:59.694548 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:59.694235    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bjn2w" event={"ID":"5e694bf8-4bfc-49c4-82e1-6b46739868cc","Type":"ContainerStarted","Data":"81f2ed236442497679dca977e4140c12cca5b79bb24866434b4bebf0b3a56ec8"}
Jul 17 04:59:59.714105 ip-10-0-132-19 kubenswrapper[2562]: I0717 04:59:59.714052    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bjn2w" podStartSLOduration=1.557589034 podStartE2EDuration="1.714038169s" totalImagesPullingTime="156.449135ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 04:59:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 04:59:59.711733025 +0000 UTC m=+587.395326683" watchObservedRunningTime="2026-07-17 04:59:59.714038169 +0000 UTC m=+587.397631825"
Jul 17 05:00:01.462870 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:00:01.462821    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b"
Jul 17 05:00:01.463295 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:00:01.462895    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b"
Jul 17 05:00:01.475497 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:00:01.475477    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b"
Jul 17 05:00:01.715774 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:00:01.715686    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b"
Jul 17 05:00:08.940514 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:00:08.940474    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bjn2w"
Jul 17 05:00:08.940908 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:00:08.940540    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bjn2w"
Jul 17 05:00:08.952819 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:00:08.952794    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bjn2w"
Jul 17 05:00:09.751294 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:00:09.751261    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bjn2w"
Jul 17 05:00:12.750709 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:00:12.750680    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tx9gj_c1baeee2-350c-43c5-9a04-fb34378edb5c/ovn-acl-logging/0.log"
Jul 17 05:00:12.753567 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:00:12.753545    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tx9gj_c1baeee2-350c-43c5-9a04-fb34378edb5c/ovn-acl-logging/0.log"
Jul 17 05:03:21.279858 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:03:21.279822    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-859ff64978-lzph8"]
Jul 17 05:03:21.280489 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:03:21.280044    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-859ff64978-lzph8" podUID="b63a7320-d527-4f4c-8d12-ad1852089444" containerName="manager" containerID="cri-o://add479b236792062b096b650a52c6996a0c962693ecb1e67dc172279dc2ddca3" gracePeriod=30
Jul 17 05:03:21.523170 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:03:21.523145    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-859ff64978-lzph8"
Jul 17 05:03:21.575853 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:03:21.575815    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9d66x\" (UniqueName: \"kubernetes.io/projected/b63a7320-d527-4f4c-8d12-ad1852089444-kube-api-access-9d66x\") pod \"b63a7320-d527-4f4c-8d12-ad1852089444\" (UID: \"b63a7320-d527-4f4c-8d12-ad1852089444\") "
Jul 17 05:03:21.576056 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:03:21.575984    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b63a7320-d527-4f4c-8d12-ad1852089444-webhook-cert\") pod \"b63a7320-d527-4f4c-8d12-ad1852089444\" (UID: \"b63a7320-d527-4f4c-8d12-ad1852089444\") "
Jul 17 05:03:21.578057 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:03:21.578026    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b63a7320-d527-4f4c-8d12-ad1852089444-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "b63a7320-d527-4f4c-8d12-ad1852089444" (UID: "b63a7320-d527-4f4c-8d12-ad1852089444"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 05:03:21.578172 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:03:21.578094    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b63a7320-d527-4f4c-8d12-ad1852089444-kube-api-access-9d66x" (OuterVolumeSpecName: "kube-api-access-9d66x") pod "b63a7320-d527-4f4c-8d12-ad1852089444" (UID: "b63a7320-d527-4f4c-8d12-ad1852089444"). InnerVolumeSpecName "kube-api-access-9d66x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:03:21.662089 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:03:21.662055    2562 generic.go:356] "Generic (PLEG): container finished" podID="b63a7320-d527-4f4c-8d12-ad1852089444" containerID="add479b236792062b096b650a52c6996a0c962693ecb1e67dc172279dc2ddca3" exitCode=0
Jul 17 05:03:21.662263 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:03:21.662105    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-859ff64978-lzph8"
Jul 17 05:03:21.662263 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:03:21.662152    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-859ff64978-lzph8" event={"ID":"b63a7320-d527-4f4c-8d12-ad1852089444","Type":"ContainerDied","Data":"add479b236792062b096b650a52c6996a0c962693ecb1e67dc172279dc2ddca3"}
Jul 17 05:03:21.662263 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:03:21.662204    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-859ff64978-lzph8" event={"ID":"b63a7320-d527-4f4c-8d12-ad1852089444","Type":"ContainerDied","Data":"f31b27c5e7dad40ad7c246e6cda90c838a5ac59d8b04ea732ffd4f38cb7f7f34"}
Jul 17 05:03:21.662263 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:03:21.662227    2562 scope.go:117] "RemoveContainer" containerID="add479b236792062b096b650a52c6996a0c962693ecb1e67dc172279dc2ddca3"
Jul 17 05:03:21.671667 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:03:21.671646    2562 scope.go:117] "RemoveContainer" containerID="add479b236792062b096b650a52c6996a0c962693ecb1e67dc172279dc2ddca3"
Jul 17 05:03:21.671944 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:03:21.671929    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"add479b236792062b096b650a52c6996a0c962693ecb1e67dc172279dc2ddca3\": container with ID starting with add479b236792062b096b650a52c6996a0c962693ecb1e67dc172279dc2ddca3 not found: ID does not exist" containerID="add479b236792062b096b650a52c6996a0c962693ecb1e67dc172279dc2ddca3"
Jul 17 05:03:21.672044 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:03:21.671954    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"add479b236792062b096b650a52c6996a0c962693ecb1e67dc172279dc2ddca3"} err="failed to get container status \"add479b236792062b096b650a52c6996a0c962693ecb1e67dc172279dc2ddca3\": rpc error: code = NotFound desc = could not find container \"add479b236792062b096b650a52c6996a0c962693ecb1e67dc172279dc2ddca3\": container with ID starting with add479b236792062b096b650a52c6996a0c962693ecb1e67dc172279dc2ddca3 not found: ID does not exist"
Jul 17 05:03:21.677370 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:03:21.677350    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9d66x\" (UniqueName: \"kubernetes.io/projected/b63a7320-d527-4f4c-8d12-ad1852089444-kube-api-access-9d66x\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:03:21.677429 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:03:21.677375    2562 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b63a7320-d527-4f4c-8d12-ad1852089444-webhook-cert\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:03:21.683189 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:03:21.683163    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-859ff64978-lzph8"]
Jul 17 05:03:21.685451 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:03:21.685427    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-859ff64978-lzph8"]
Jul 17 05:03:22.884681 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:03:22.884644    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b63a7320-d527-4f4c-8d12-ad1852089444" path="/var/lib/kubelet/pods/b63a7320-d527-4f4c-8d12-ad1852089444/volumes"
Jul 17 05:04:22.498549 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:22.498511    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-859ff64978-f8cft"]
Jul 17 05:04:22.498945 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:22.498937    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b63a7320-d527-4f4c-8d12-ad1852089444" containerName="manager"
Jul 17 05:04:22.499020 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:22.498949    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b63a7320-d527-4f4c-8d12-ad1852089444" containerName="manager"
Jul 17 05:04:22.499056 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:22.499026    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b63a7320-d527-4f4c-8d12-ad1852089444" containerName="manager"
Jul 17 05:04:22.501667 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:22.501650    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-859ff64978-f8cft"
Jul 17 05:04:22.504301 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:22.504280    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 17 05:04:22.505372 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:22.505356    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-5j47z\""
Jul 17 05:04:22.505460 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:22.505398    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 17 05:04:22.510320 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:22.510295    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-859ff64978-f8cft"]
Jul 17 05:04:22.636816 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:22.636777    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvrhq\" (UniqueName: \"kubernetes.io/projected/189b6f2f-e32f-46a8-bcc7-557e8f4f0121-kube-api-access-tvrhq\") pod \"maas-controller-859ff64978-f8cft\" (UID: \"189b6f2f-e32f-46a8-bcc7-557e8f4f0121\") " pod="opendatahub/maas-controller-859ff64978-f8cft"
Jul 17 05:04:22.637027 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:22.636909    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/189b6f2f-e32f-46a8-bcc7-557e8f4f0121-webhook-cert\") pod \"maas-controller-859ff64978-f8cft\" (UID: \"189b6f2f-e32f-46a8-bcc7-557e8f4f0121\") " pod="opendatahub/maas-controller-859ff64978-f8cft"
Jul 17 05:04:22.737503 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:22.737467    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/189b6f2f-e32f-46a8-bcc7-557e8f4f0121-webhook-cert\") pod \"maas-controller-859ff64978-f8cft\" (UID: \"189b6f2f-e32f-46a8-bcc7-557e8f4f0121\") " pod="opendatahub/maas-controller-859ff64978-f8cft"
Jul 17 05:04:22.737678 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:22.737548    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvrhq\" (UniqueName: \"kubernetes.io/projected/189b6f2f-e32f-46a8-bcc7-557e8f4f0121-kube-api-access-tvrhq\") pod \"maas-controller-859ff64978-f8cft\" (UID: \"189b6f2f-e32f-46a8-bcc7-557e8f4f0121\") " pod="opendatahub/maas-controller-859ff64978-f8cft"
Jul 17 05:04:22.740223 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:22.740198    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/189b6f2f-e32f-46a8-bcc7-557e8f4f0121-webhook-cert\") pod \"maas-controller-859ff64978-f8cft\" (UID: \"189b6f2f-e32f-46a8-bcc7-557e8f4f0121\") " pod="opendatahub/maas-controller-859ff64978-f8cft"
Jul 17 05:04:22.746013 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:22.745991    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvrhq\" (UniqueName: \"kubernetes.io/projected/189b6f2f-e32f-46a8-bcc7-557e8f4f0121-kube-api-access-tvrhq\") pod \"maas-controller-859ff64978-f8cft\" (UID: \"189b6f2f-e32f-46a8-bcc7-557e8f4f0121\") " pod="opendatahub/maas-controller-859ff64978-f8cft"
Jul 17 05:04:22.813762 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:22.813671    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-859ff64978-f8cft"
Jul 17 05:04:23.155673 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:23.155641    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-859ff64978-f8cft"]
Jul 17 05:04:23.156691 ip-10-0-132-19 kubenswrapper[2562]: W0717 05:04:23.156657    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod189b6f2f_e32f_46a8_bcc7_557e8f4f0121.slice/crio-391d095f97598bb72f789a119de3ac8be5fabdba8d4ace391456de98c4b220c6 WatchSource:0}: Error finding container 391d095f97598bb72f789a119de3ac8be5fabdba8d4ace391456de98c4b220c6: Status 404 returned error can't find the container with id 391d095f97598bb72f789a119de3ac8be5fabdba8d4ace391456de98c4b220c6
Jul 17 05:04:23.158293 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:23.158136    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 05:04:23.964695 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:23.964659    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-859ff64978-f8cft" event={"ID":"189b6f2f-e32f-46a8-bcc7-557e8f4f0121","Type":"ContainerStarted","Data":"b9c015ade4c60e04c2a989b2bacb1f20b64fd5081bbc8bf5594a5ed3722f1f4a"}
Jul 17 05:04:23.965111 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:23.964715    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-859ff64978-f8cft" event={"ID":"189b6f2f-e32f-46a8-bcc7-557e8f4f0121","Type":"ContainerStarted","Data":"391d095f97598bb72f789a119de3ac8be5fabdba8d4ace391456de98c4b220c6"}
Jul 17 05:04:23.965111 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:23.964773    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-859ff64978-f8cft"
Jul 17 05:04:23.982711 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:23.982608    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-859ff64978-f8cft" podStartSLOduration=1.422930973 podStartE2EDuration="1.982588344s" totalImagesPullingTime="559.657371ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:04:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:04:23.979945372 +0000 UTC m=+851.663539030" watchObservedRunningTime="2026-07-17 05:04:23.982588344 +0000 UTC m=+851.666182003"
Jul 17 05:04:34.971438 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:34.971405    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-859ff64978-f8cft"
Jul 17 05:04:39.592103 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:39.592065    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-859ff64978-f8cft"]
Jul 17 05:04:39.592601 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:39.592288    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-859ff64978-f8cft" podUID="189b6f2f-e32f-46a8-bcc7-557e8f4f0121" containerName="manager" containerID="cri-o://b9c015ade4c60e04c2a989b2bacb1f20b64fd5081bbc8bf5594a5ed3722f1f4a" gracePeriod=30
Jul 17 05:04:39.841860 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:39.841836    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-859ff64978-f8cft"
Jul 17 05:04:39.990251 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:39.990171    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tvrhq\" (UniqueName: \"kubernetes.io/projected/189b6f2f-e32f-46a8-bcc7-557e8f4f0121-kube-api-access-tvrhq\") pod \"189b6f2f-e32f-46a8-bcc7-557e8f4f0121\" (UID: \"189b6f2f-e32f-46a8-bcc7-557e8f4f0121\") "
Jul 17 05:04:39.990251 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:39.990206    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/189b6f2f-e32f-46a8-bcc7-557e8f4f0121-webhook-cert\") pod \"189b6f2f-e32f-46a8-bcc7-557e8f4f0121\" (UID: \"189b6f2f-e32f-46a8-bcc7-557e8f4f0121\") "
Jul 17 05:04:39.992384 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:39.992354    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/189b6f2f-e32f-46a8-bcc7-557e8f4f0121-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "189b6f2f-e32f-46a8-bcc7-557e8f4f0121" (UID: "189b6f2f-e32f-46a8-bcc7-557e8f4f0121"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 05:04:39.992495 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:39.992407    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/189b6f2f-e32f-46a8-bcc7-557e8f4f0121-kube-api-access-tvrhq" (OuterVolumeSpecName: "kube-api-access-tvrhq") pod "189b6f2f-e32f-46a8-bcc7-557e8f4f0121" (UID: "189b6f2f-e32f-46a8-bcc7-557e8f4f0121"). InnerVolumeSpecName "kube-api-access-tvrhq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:04:40.040289 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:40.040247    2562 generic.go:356] "Generic (PLEG): container finished" podID="189b6f2f-e32f-46a8-bcc7-557e8f4f0121" containerID="b9c015ade4c60e04c2a989b2bacb1f20b64fd5081bbc8bf5594a5ed3722f1f4a" exitCode=0
Jul 17 05:04:40.040456 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:40.040296    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-859ff64978-f8cft"
Jul 17 05:04:40.040456 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:40.040326    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-859ff64978-f8cft" event={"ID":"189b6f2f-e32f-46a8-bcc7-557e8f4f0121","Type":"ContainerDied","Data":"b9c015ade4c60e04c2a989b2bacb1f20b64fd5081bbc8bf5594a5ed3722f1f4a"}
Jul 17 05:04:40.040456 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:40.040379    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-859ff64978-f8cft" event={"ID":"189b6f2f-e32f-46a8-bcc7-557e8f4f0121","Type":"ContainerDied","Data":"391d095f97598bb72f789a119de3ac8be5fabdba8d4ace391456de98c4b220c6"}
Jul 17 05:04:40.040456 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:40.040402    2562 scope.go:117] "RemoveContainer" containerID="b9c015ade4c60e04c2a989b2bacb1f20b64fd5081bbc8bf5594a5ed3722f1f4a"
Jul 17 05:04:40.049877 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:40.049860    2562 scope.go:117] "RemoveContainer" containerID="b9c015ade4c60e04c2a989b2bacb1f20b64fd5081bbc8bf5594a5ed3722f1f4a"
Jul 17 05:04:40.050180 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:04:40.050148    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9c015ade4c60e04c2a989b2bacb1f20b64fd5081bbc8bf5594a5ed3722f1f4a\": container with ID starting with b9c015ade4c60e04c2a989b2bacb1f20b64fd5081bbc8bf5594a5ed3722f1f4a not found: ID does not exist" containerID="b9c015ade4c60e04c2a989b2bacb1f20b64fd5081bbc8bf5594a5ed3722f1f4a"
Jul 17 05:04:40.050259 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:40.050189    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9c015ade4c60e04c2a989b2bacb1f20b64fd5081bbc8bf5594a5ed3722f1f4a"} err="failed to get container status \"b9c015ade4c60e04c2a989b2bacb1f20b64fd5081bbc8bf5594a5ed3722f1f4a\": rpc error: code = NotFound desc = could not find container \"b9c015ade4c60e04c2a989b2bacb1f20b64fd5081bbc8bf5594a5ed3722f1f4a\": container with ID starting with b9c015ade4c60e04c2a989b2bacb1f20b64fd5081bbc8bf5594a5ed3722f1f4a not found: ID does not exist"
Jul 17 05:04:40.061695 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:40.061668    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-859ff64978-f8cft"]
Jul 17 05:04:40.065520 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:40.065490    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-859ff64978-f8cft"]
Jul 17 05:04:40.091123 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:40.091094    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tvrhq\" (UniqueName: \"kubernetes.io/projected/189b6f2f-e32f-46a8-bcc7-557e8f4f0121-kube-api-access-tvrhq\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:04:40.091123 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:40.091124    2562 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/189b6f2f-e32f-46a8-bcc7-557e8f4f0121-webhook-cert\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:04:40.827659 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:40.827625    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-859ff64978-dq8x9"]
Jul 17 05:04:40.828251 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:40.828024    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="189b6f2f-e32f-46a8-bcc7-557e8f4f0121" containerName="manager"
Jul 17 05:04:40.828251 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:40.828036    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="189b6f2f-e32f-46a8-bcc7-557e8f4f0121" containerName="manager"
Jul 17 05:04:40.828251 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:40.828126    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="189b6f2f-e32f-46a8-bcc7-557e8f4f0121" containerName="manager"
Jul 17 05:04:40.830855 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:40.830838    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-859ff64978-dq8x9"
Jul 17 05:04:40.834440 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:40.834422    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 17 05:04:40.834554 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:40.834424    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 17 05:04:40.834554 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:40.834458    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-5j47z\""
Jul 17 05:04:40.840798 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:40.840763    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-859ff64978-dq8x9"]
Jul 17 05:04:40.884829 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:40.884795    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="189b6f2f-e32f-46a8-bcc7-557e8f4f0121" path="/var/lib/kubelet/pods/189b6f2f-e32f-46a8-bcc7-557e8f4f0121/volumes"
Jul 17 05:04:41.000076 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:41.000039    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8f3b63a1-b0a2-42d4-ab11-11ad32132d44-webhook-cert\") pod \"maas-controller-859ff64978-dq8x9\" (UID: \"8f3b63a1-b0a2-42d4-ab11-11ad32132d44\") " pod="opendatahub/maas-controller-859ff64978-dq8x9"
Jul 17 05:04:41.000272 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:41.000096    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68nhf\" (UniqueName: \"kubernetes.io/projected/8f3b63a1-b0a2-42d4-ab11-11ad32132d44-kube-api-access-68nhf\") pod \"maas-controller-859ff64978-dq8x9\" (UID: \"8f3b63a1-b0a2-42d4-ab11-11ad32132d44\") " pod="opendatahub/maas-controller-859ff64978-dq8x9"
Jul 17 05:04:41.101104 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:41.101010    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68nhf\" (UniqueName: \"kubernetes.io/projected/8f3b63a1-b0a2-42d4-ab11-11ad32132d44-kube-api-access-68nhf\") pod \"maas-controller-859ff64978-dq8x9\" (UID: \"8f3b63a1-b0a2-42d4-ab11-11ad32132d44\") " pod="opendatahub/maas-controller-859ff64978-dq8x9"
Jul 17 05:04:41.101314 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:41.101135    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8f3b63a1-b0a2-42d4-ab11-11ad32132d44-webhook-cert\") pod \"maas-controller-859ff64978-dq8x9\" (UID: \"8f3b63a1-b0a2-42d4-ab11-11ad32132d44\") " pod="opendatahub/maas-controller-859ff64978-dq8x9"
Jul 17 05:04:41.103628 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:41.103609    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8f3b63a1-b0a2-42d4-ab11-11ad32132d44-webhook-cert\") pod \"maas-controller-859ff64978-dq8x9\" (UID: \"8f3b63a1-b0a2-42d4-ab11-11ad32132d44\") " pod="opendatahub/maas-controller-859ff64978-dq8x9"
Jul 17 05:04:41.108742 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:41.108725    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68nhf\" (UniqueName: \"kubernetes.io/projected/8f3b63a1-b0a2-42d4-ab11-11ad32132d44-kube-api-access-68nhf\") pod \"maas-controller-859ff64978-dq8x9\" (UID: \"8f3b63a1-b0a2-42d4-ab11-11ad32132d44\") " pod="opendatahub/maas-controller-859ff64978-dq8x9"
Jul 17 05:04:41.142043 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:41.142010    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-859ff64978-dq8x9"
Jul 17 05:04:41.274994 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:41.274949    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-859ff64978-dq8x9"]
Jul 17 05:04:41.277136 ip-10-0-132-19 kubenswrapper[2562]: W0717 05:04:41.277111    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f3b63a1_b0a2_42d4_ab11_11ad32132d44.slice/crio-ab62eebcacceb0c0fc6817e99624c47cd4591db6a540adc3c243edbfd4640eea WatchSource:0}: Error finding container ab62eebcacceb0c0fc6817e99624c47cd4591db6a540adc3c243edbfd4640eea: Status 404 returned error can't find the container with id ab62eebcacceb0c0fc6817e99624c47cd4591db6a540adc3c243edbfd4640eea
Jul 17 05:04:42.054217 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:42.054117    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-859ff64978-dq8x9" event={"ID":"8f3b63a1-b0a2-42d4-ab11-11ad32132d44","Type":"ContainerStarted","Data":"92591ca2d2f670e1f459492bdd4c29573e03bb48e3c47e4554fbd3a8bea57e67"}
Jul 17 05:04:42.054567 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:42.054216    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-859ff64978-dq8x9" event={"ID":"8f3b63a1-b0a2-42d4-ab11-11ad32132d44","Type":"ContainerStarted","Data":"ab62eebcacceb0c0fc6817e99624c47cd4591db6a540adc3c243edbfd4640eea"}
Jul 17 05:04:42.054567 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:42.054247    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-859ff64978-dq8x9"
Jul 17 05:04:42.070208 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:42.070147    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-859ff64978-dq8x9" podStartSLOduration=1.712835742 podStartE2EDuration="2.070127094s" totalImagesPullingTime="357.291352ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:04:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:04:42.069300802 +0000 UTC m=+869.752894485" watchObservedRunningTime="2026-07-17 05:04:42.070127094 +0000 UTC m=+869.753720754"
Jul 17 05:04:53.061618 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:04:53.061582    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-859ff64978-dq8x9"
Jul 17 05:05:12.787126 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:05:12.787094    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tx9gj_c1baeee2-350c-43c5-9a04-fb34378edb5c/ovn-acl-logging/0.log"
Jul 17 05:05:12.788284 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:05:12.788248    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tx9gj_c1baeee2-350c-43c5-9a04-fb34378edb5c/ovn-acl-logging/0.log"
Jul 17 05:10:12.821406 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:10:12.821369    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tx9gj_c1baeee2-350c-43c5-9a04-fb34378edb5c/ovn-acl-logging/0.log"
Jul 17 05:10:12.822014 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:10:12.821760    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tx9gj_c1baeee2-350c-43c5-9a04-fb34378edb5c/ovn-acl-logging/0.log"
Jul 17 05:15:00.141974 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:15:00.141935    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737755-6zrtq"]
Jul 17 05:15:00.145415 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:15:00.145394    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737755-6zrtq"
Jul 17 05:15:00.147924 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:15:00.147900    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qh8zj\""
Jul 17 05:15:00.161284 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:15:00.161244    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737755-6zrtq"]
Jul 17 05:15:00.166209 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:15:00.166171    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737755-6zrtq"
Jul 17 05:15:00.307754 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:15:00.307728    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737755-6zrtq"]
Jul 17 05:15:00.309796 ip-10-0-132-19 kubenswrapper[2562]: W0717 05:15:00.309762    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9f3d10c_1add_4371_984d_56d64050ca2e.slice/crio-f0e8d8dd754ab45a22be822bf88c2a91e1045bd3ba99aa93012a285e9de4b065 WatchSource:0}: Error finding container f0e8d8dd754ab45a22be822bf88c2a91e1045bd3ba99aa93012a285e9de4b065: Status 404 returned error can't find the container with id f0e8d8dd754ab45a22be822bf88c2a91e1045bd3ba99aa93012a285e9de4b065
Jul 17 05:15:00.311693 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:15:00.311679    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 05:15:01.062177 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:15:01.062127    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737755-6zrtq" event={"ID":"c9f3d10c-1add-4371-984d-56d64050ca2e","Type":"ContainerStarted","Data":"f0e8d8dd754ab45a22be822bf88c2a91e1045bd3ba99aa93012a285e9de4b065"}
Jul 17 05:15:03.072765 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:15:03.072721    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737755-6zrtq" event={"ID":"c9f3d10c-1add-4371-984d-56d64050ca2e","Type":"ContainerStarted","Data":"0ce75646888d6ef3b0275d9425507bfe1a20490e9620bf6dec13e007cdbcbacd"}
Jul 17 05:15:03.088751 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:15:03.088696    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737755-6zrtq" podStartSLOduration=1.024704736 podStartE2EDuration="3.08867937s" totalImagesPullingTime="2.063974634s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:15:03.087714667 +0000 UTC m=+1490.771308325" watchObservedRunningTime="2026-07-17 05:15:03.08867937 +0000 UTC m=+1490.772273026"
Jul 17 05:15:04.079294 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:15:04.079258    2562 generic.go:356] "Generic (PLEG): container finished" podID="c9f3d10c-1add-4371-984d-56d64050ca2e" containerID="0ce75646888d6ef3b0275d9425507bfe1a20490e9620bf6dec13e007cdbcbacd" exitCode=0
Jul 17 05:15:04.079698 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:15:04.079327    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737755-6zrtq" event={"ID":"c9f3d10c-1add-4371-984d-56d64050ca2e","Type":"ContainerDied","Data":"0ce75646888d6ef3b0275d9425507bfe1a20490e9620bf6dec13e007cdbcbacd"}
Jul 17 05:15:05.219692 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:15:05.219655    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737755-6zrtq"
Jul 17 05:15:06.089801 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:15:06.089769    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737755-6zrtq"
Jul 17 05:15:06.090016 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:15:06.089811    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737755-6zrtq" event={"ID":"c9f3d10c-1add-4371-984d-56d64050ca2e","Type":"ContainerDied","Data":"f0e8d8dd754ab45a22be822bf88c2a91e1045bd3ba99aa93012a285e9de4b065"}
Jul 17 05:15:06.090016 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:15:06.089861    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f0e8d8dd754ab45a22be822bf88c2a91e1045bd3ba99aa93012a285e9de4b065"
Jul 17 05:15:12.859139 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:15:12.859106    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tx9gj_c1baeee2-350c-43c5-9a04-fb34378edb5c/ovn-acl-logging/0.log"
Jul 17 05:15:12.860100 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:15:12.860076    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tx9gj_c1baeee2-350c-43c5-9a04-fb34378edb5c/ovn-acl-logging/0.log"
Jul 17 05:16:33.823051 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:33.823007    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t"]
Jul 17 05:16:33.823655 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:33.823637    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9f3d10c-1add-4371-984d-56d64050ca2e" containerName="cleanup"
Jul 17 05:16:33.823655 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:33.823656    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9f3d10c-1add-4371-984d-56d64050ca2e" containerName="cleanup"
Jul 17 05:16:33.823794 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:33.823758    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9f3d10c-1add-4371-984d-56d64050ca2e" containerName="cleanup"
Jul 17 05:16:33.826281 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:33.826261    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t"
Jul 17 05:16:33.829352 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:33.829330    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-4e221674-openshift-default-dockercfg-z5f6g\""
Jul 17 05:16:33.838996 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:33.838951    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t"]
Jul 17 05:16:33.903790 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:33.903750    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c14a1668-297d-4028-96d3-e5be393a0c37-istio-podinfo\") pod \"e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t\" (UID: \"c14a1668-297d-4028-96d3-e5be393a0c37\") " pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t"
Jul 17 05:16:33.903999 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:33.903797    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c14a1668-297d-4028-96d3-e5be393a0c37-credential-socket\") pod \"e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t\" (UID: \"c14a1668-297d-4028-96d3-e5be393a0c37\") " pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t"
Jul 17 05:16:33.903999 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:33.903820    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c14a1668-297d-4028-96d3-e5be393a0c37-istio-envoy\") pod \"e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t\" (UID: \"c14a1668-297d-4028-96d3-e5be393a0c37\") " pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t"
Jul 17 05:16:33.903999 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:33.903934    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c14a1668-297d-4028-96d3-e5be393a0c37-workload-socket\") pod \"e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t\" (UID: \"c14a1668-297d-4028-96d3-e5be393a0c37\") " pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t"
Jul 17 05:16:33.903999 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:33.903993    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c14a1668-297d-4028-96d3-e5be393a0c37-istio-data\") pod \"e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t\" (UID: \"c14a1668-297d-4028-96d3-e5be393a0c37\") " pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t"
Jul 17 05:16:33.904177 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:33.904046    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c14a1668-297d-4028-96d3-e5be393a0c37-istio-token\") pod \"e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t\" (UID: \"c14a1668-297d-4028-96d3-e5be393a0c37\") " pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t"
Jul 17 05:16:33.904177 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:33.904137    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c14a1668-297d-4028-96d3-e5be393a0c37-workload-certs\") pod \"e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t\" (UID: \"c14a1668-297d-4028-96d3-e5be393a0c37\") " pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t"
Jul 17 05:16:33.904177 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:33.904157    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c14a1668-297d-4028-96d3-e5be393a0c37-istiod-ca-cert\") pod \"e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t\" (UID: \"c14a1668-297d-4028-96d3-e5be393a0c37\") " pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t"
Jul 17 05:16:33.904291 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:33.904179    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7j9s\" (UniqueName: \"kubernetes.io/projected/c14a1668-297d-4028-96d3-e5be393a0c37-kube-api-access-m7j9s\") pod \"e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t\" (UID: \"c14a1668-297d-4028-96d3-e5be393a0c37\") " pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t"
Jul 17 05:16:34.005330 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:34.005290    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c14a1668-297d-4028-96d3-e5be393a0c37-workload-certs\") pod \"e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t\" (UID: \"c14a1668-297d-4028-96d3-e5be393a0c37\") " pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t"
Jul 17 05:16:34.005330 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:34.005331    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c14a1668-297d-4028-96d3-e5be393a0c37-istiod-ca-cert\") pod \"e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t\" (UID: \"c14a1668-297d-4028-96d3-e5be393a0c37\") " pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t"
Jul 17 05:16:34.005598 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:34.005354    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7j9s\" (UniqueName: \"kubernetes.io/projected/c14a1668-297d-4028-96d3-e5be393a0c37-kube-api-access-m7j9s\") pod \"e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t\" (UID: \"c14a1668-297d-4028-96d3-e5be393a0c37\") " pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t"
Jul 17 05:16:34.005598 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:34.005386    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c14a1668-297d-4028-96d3-e5be393a0c37-istio-podinfo\") pod \"e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t\" (UID: \"c14a1668-297d-4028-96d3-e5be393a0c37\") " pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t"
Jul 17 05:16:34.005598 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:34.005504    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c14a1668-297d-4028-96d3-e5be393a0c37-credential-socket\") pod \"e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t\" (UID: \"c14a1668-297d-4028-96d3-e5be393a0c37\") " pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t"
Jul 17 05:16:34.005598 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:34.005538    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c14a1668-297d-4028-96d3-e5be393a0c37-istio-envoy\") pod \"e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t\" (UID: \"c14a1668-297d-4028-96d3-e5be393a0c37\") " pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t"
Jul 17 05:16:34.005598 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:34.005567    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c14a1668-297d-4028-96d3-e5be393a0c37-workload-socket\") pod \"e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t\" (UID: \"c14a1668-297d-4028-96d3-e5be393a0c37\") " pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t"
Jul 17 05:16:34.005598 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:34.005591    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c14a1668-297d-4028-96d3-e5be393a0c37-istio-data\") pod \"e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t\" (UID: \"c14a1668-297d-4028-96d3-e5be393a0c37\") " pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t"
Jul 17 05:16:34.005903 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:34.005626    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c14a1668-297d-4028-96d3-e5be393a0c37-istio-token\") pod \"e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t\" (UID: \"c14a1668-297d-4028-96d3-e5be393a0c37\") " pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t"
Jul 17 05:16:34.005903 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:34.005805    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c14a1668-297d-4028-96d3-e5be393a0c37-workload-certs\") pod \"e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t\" (UID: \"c14a1668-297d-4028-96d3-e5be393a0c37\") " pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t"
Jul 17 05:16:34.006120 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:34.006100    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c14a1668-297d-4028-96d3-e5be393a0c37-credential-socket\") pod \"e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t\" (UID: \"c14a1668-297d-4028-96d3-e5be393a0c37\") " pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t"
Jul 17 05:16:34.006319 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:34.006299    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c14a1668-297d-4028-96d3-e5be393a0c37-workload-socket\") pod \"e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t\" (UID: \"c14a1668-297d-4028-96d3-e5be393a0c37\") " pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t"
Jul 17 05:16:34.006389 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:34.006367    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c14a1668-297d-4028-96d3-e5be393a0c37-istio-data\") pod \"e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t\" (UID: \"c14a1668-297d-4028-96d3-e5be393a0c37\") " pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t"
Jul 17 05:16:34.006709 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:34.006686    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c14a1668-297d-4028-96d3-e5be393a0c37-istiod-ca-cert\") pod \"e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t\" (UID: \"c14a1668-297d-4028-96d3-e5be393a0c37\") " pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t"
Jul 17 05:16:34.007921 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:34.007899    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c14a1668-297d-4028-96d3-e5be393a0c37-istio-envoy\") pod \"e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t\" (UID: \"c14a1668-297d-4028-96d3-e5be393a0c37\") " pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t"
Jul 17 05:16:34.008079 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:34.008064    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c14a1668-297d-4028-96d3-e5be393a0c37-istio-podinfo\") pod \"e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t\" (UID: \"c14a1668-297d-4028-96d3-e5be393a0c37\") " pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t"
Jul 17 05:16:34.020108 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:34.020084    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c14a1668-297d-4028-96d3-e5be393a0c37-istio-token\") pod \"e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t\" (UID: \"c14a1668-297d-4028-96d3-e5be393a0c37\") " pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t"
Jul 17 05:16:34.020240 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:34.020224    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7j9s\" (UniqueName: \"kubernetes.io/projected/c14a1668-297d-4028-96d3-e5be393a0c37-kube-api-access-m7j9s\") pod \"e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t\" (UID: \"c14a1668-297d-4028-96d3-e5be393a0c37\") " pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t"
Jul 17 05:16:34.138546 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:34.138456    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t"
Jul 17 05:16:34.277387 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:34.277366    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t"]
Jul 17 05:16:34.279791 ip-10-0-132-19 kubenswrapper[2562]: W0717 05:16:34.279765    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc14a1668_297d_4028_96d3_e5be393a0c37.slice/crio-190c051664d944f279b7a0beabad964e215a29636b061e48d199e1a577873bda WatchSource:0}: Error finding container 190c051664d944f279b7a0beabad964e215a29636b061e48d199e1a577873bda: Status 404 returned error can't find the container with id 190c051664d944f279b7a0beabad964e215a29636b061e48d199e1a577873bda
Jul 17 05:16:34.281832 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:34.281796    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:16:34.281917 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:34.281868    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:16:34.281917 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:34.281898    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:16:34.521680 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:34.521586    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t" event={"ID":"c14a1668-297d-4028-96d3-e5be393a0c37","Type":"ContainerStarted","Data":"55b460f7fb559c43cca666b6f5802c26edf366bafa168dab5c2509e6448a6abc"}
Jul 17 05:16:34.521680 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:34.521652    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t" event={"ID":"c14a1668-297d-4028-96d3-e5be393a0c37","Type":"ContainerStarted","Data":"190c051664d944f279b7a0beabad964e215a29636b061e48d199e1a577873bda"}
Jul 17 05:16:34.559077 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:34.559024    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t" podStartSLOduration=1.559006955 podStartE2EDuration="1.559006955s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:16:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:16:34.556471831 +0000 UTC m=+1582.240065488" watchObservedRunningTime="2026-07-17 05:16:34.559006955 +0000 UTC m=+1582.242600809"
Jul 17 05:16:35.139281 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:35.139233    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t"
Jul 17 05:16:35.144249 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:35.144221    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t"
Jul 17 05:16:35.219591 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:35.219559    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9"]
Jul 17 05:16:35.222886 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:35.222863    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9"
Jul 17 05:16:35.226145 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:35.226123    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-xwcd7\""
Jul 17 05:16:35.226283 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:35.226144    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-4e221674-serving-cert\""
Jul 17 05:16:35.235988 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:35.235942    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9"]
Jul 17 05:16:35.319458 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:35.319417    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwd6t\" (UniqueName: \"kubernetes.io/projected/0e7718a9-2705-4130-864b-6d1a9df9c5d6-kube-api-access-rwd6t\") pod \"maas-api-e2e-ait-4e221674-5ff85f6499-zdth9\" (UID: \"0e7718a9-2705-4130-864b-6d1a9df9c5d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9"
Jul 17 05:16:35.319627 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:35.319475    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e7718a9-2705-4130-864b-6d1a9df9c5d6-openshift-service-ca\") pod \"maas-api-e2e-ait-4e221674-5ff85f6499-zdth9\" (UID: \"0e7718a9-2705-4130-864b-6d1a9df9c5d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9"
Jul 17 05:16:35.319627 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:35.319551    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e7718a9-2705-4130-864b-6d1a9df9c5d6-maas-api-tls\") pod \"maas-api-e2e-ait-4e221674-5ff85f6499-zdth9\" (UID: \"0e7718a9-2705-4130-864b-6d1a9df9c5d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9"
Jul 17 05:16:35.420252 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:35.420167    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwd6t\" (UniqueName: \"kubernetes.io/projected/0e7718a9-2705-4130-864b-6d1a9df9c5d6-kube-api-access-rwd6t\") pod \"maas-api-e2e-ait-4e221674-5ff85f6499-zdth9\" (UID: \"0e7718a9-2705-4130-864b-6d1a9df9c5d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9"
Jul 17 05:16:35.420252 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:35.420215    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e7718a9-2705-4130-864b-6d1a9df9c5d6-openshift-service-ca\") pod \"maas-api-e2e-ait-4e221674-5ff85f6499-zdth9\" (UID: \"0e7718a9-2705-4130-864b-6d1a9df9c5d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9"
Jul 17 05:16:35.420252 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:35.420244    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e7718a9-2705-4130-864b-6d1a9df9c5d6-maas-api-tls\") pod \"maas-api-e2e-ait-4e221674-5ff85f6499-zdth9\" (UID: \"0e7718a9-2705-4130-864b-6d1a9df9c5d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9"
Jul 17 05:16:35.420862 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:35.420839    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e7718a9-2705-4130-864b-6d1a9df9c5d6-openshift-service-ca\") pod \"maas-api-e2e-ait-4e221674-5ff85f6499-zdth9\" (UID: \"0e7718a9-2705-4130-864b-6d1a9df9c5d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9"
Jul 17 05:16:35.422925 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:35.422906    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e7718a9-2705-4130-864b-6d1a9df9c5d6-maas-api-tls\") pod \"maas-api-e2e-ait-4e221674-5ff85f6499-zdth9\" (UID: \"0e7718a9-2705-4130-864b-6d1a9df9c5d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9"
Jul 17 05:16:35.428835 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:35.428811    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwd6t\" (UniqueName: \"kubernetes.io/projected/0e7718a9-2705-4130-864b-6d1a9df9c5d6-kube-api-access-rwd6t\") pod \"maas-api-e2e-ait-4e221674-5ff85f6499-zdth9\" (UID: \"0e7718a9-2705-4130-864b-6d1a9df9c5d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9"
Jul 17 05:16:35.526304 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:35.526270    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t"
Jul 17 05:16:35.527185 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:35.527167    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t"
Jul 17 05:16:35.534539 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:35.534517    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9"
Jul 17 05:16:35.692272 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:35.692248    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9"]
Jul 17 05:16:35.694235 ip-10-0-132-19 kubenswrapper[2562]: W0717 05:16:35.694199    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e7718a9_2705_4130_864b_6d1a9df9c5d6.slice/crio-84bfc675cac0c8ff7755259ca24c7ec8aab745a6e2d0fdf1421f00189355e93b WatchSource:0}: Error finding container 84bfc675cac0c8ff7755259ca24c7ec8aab745a6e2d0fdf1421f00189355e93b: Status 404 returned error can't find the container with id 84bfc675cac0c8ff7755259ca24c7ec8aab745a6e2d0fdf1421f00189355e93b
Jul 17 05:16:36.532174 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:36.532136    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9" event={"ID":"0e7718a9-2705-4130-864b-6d1a9df9c5d6","Type":"ContainerStarted","Data":"3aa611a2dfc7f1a75cb14d18ba66161b48f2f09675c141fc6e33a9ec7ee52e2e"}
Jul 17 05:16:36.532596 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:36.532196    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9" event={"ID":"0e7718a9-2705-4130-864b-6d1a9df9c5d6","Type":"ContainerStarted","Data":"84bfc675cac0c8ff7755259ca24c7ec8aab745a6e2d0fdf1421f00189355e93b"}
Jul 17 05:16:36.532596 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:36.532235    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9"
Jul 17 05:16:36.550987 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:36.550923    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9" podStartSLOduration=0.823361128 podStartE2EDuration="1.550906438s" totalImagesPullingTime="727.54531ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:16:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:16:36.54988654 +0000 UTC m=+1584.233480198" watchObservedRunningTime="2026-07-17 05:16:36.550906438 +0000 UTC m=+1584.234500098"
Jul 17 05:16:37.539472 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:37.539432    2562 generic.go:356] "Generic (PLEG): container finished" podID="0e7718a9-2705-4130-864b-6d1a9df9c5d6" containerID="3aa611a2dfc7f1a75cb14d18ba66161b48f2f09675c141fc6e33a9ec7ee52e2e" exitCode=1
Jul 17 05:16:37.539942 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:37.539515    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9" event={"ID":"0e7718a9-2705-4130-864b-6d1a9df9c5d6","Type":"ContainerDied","Data":"3aa611a2dfc7f1a75cb14d18ba66161b48f2f09675c141fc6e33a9ec7ee52e2e"}
Jul 17 05:16:37.539942 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:37.539726    2562 scope.go:117] "RemoveContainer" containerID="3aa611a2dfc7f1a75cb14d18ba66161b48f2f09675c141fc6e33a9ec7ee52e2e"
Jul 17 05:16:38.546773 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:38.546694    2562 generic.go:356] "Generic (PLEG): container finished" podID="0e7718a9-2705-4130-864b-6d1a9df9c5d6" containerID="0a4ca3bced6f6506ed772aadc053fabe1a97fd96b26141ce2b3247a5fc97b6f1" exitCode=1
Jul 17 05:16:38.547174 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:38.546776    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9" event={"ID":"0e7718a9-2705-4130-864b-6d1a9df9c5d6","Type":"ContainerDied","Data":"0a4ca3bced6f6506ed772aadc053fabe1a97fd96b26141ce2b3247a5fc97b6f1"}
Jul 17 05:16:38.547174 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:38.546832    2562 scope.go:117] "RemoveContainer" containerID="3aa611a2dfc7f1a75cb14d18ba66161b48f2f09675c141fc6e33a9ec7ee52e2e"
Jul 17 05:16:38.547174 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:38.547096    2562 scope.go:117] "RemoveContainer" containerID="0a4ca3bced6f6506ed772aadc053fabe1a97fd96b26141ce2b3247a5fc97b6f1"
Jul 17 05:16:38.547364 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:16:38.547344    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-4e221674-5ff85f6499-zdth9_odh-ai-gateway-infra(0e7718a9-2705-4130-864b-6d1a9df9c5d6)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9" podUID="0e7718a9-2705-4130-864b-6d1a9df9c5d6"
Jul 17 05:16:39.553975 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:39.553944    2562 scope.go:117] "RemoveContainer" containerID="0a4ca3bced6f6506ed772aadc053fabe1a97fd96b26141ce2b3247a5fc97b6f1"
Jul 17 05:16:39.554377 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:16:39.554163    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-4e221674-5ff85f6499-zdth9_odh-ai-gateway-infra(0e7718a9-2705-4130-864b-6d1a9df9c5d6)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9" podUID="0e7718a9-2705-4130-864b-6d1a9df9c5d6"
Jul 17 05:16:41.533151 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:41.533105    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9"
Jul 17 05:16:41.533555 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:41.533523    2562 scope.go:117] "RemoveContainer" containerID="0a4ca3bced6f6506ed772aadc053fabe1a97fd96b26141ce2b3247a5fc97b6f1"
Jul 17 05:16:41.533718 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:16:41.533698    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-4e221674-5ff85f6499-zdth9_odh-ai-gateway-infra(0e7718a9-2705-4130-864b-6d1a9df9c5d6)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9" podUID="0e7718a9-2705-4130-864b-6d1a9df9c5d6"
Jul 17 05:16:45.534772 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:45.534739    2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9"
Jul 17 05:16:45.535276 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:45.535253    2562 scope.go:117] "RemoveContainer" containerID="0a4ca3bced6f6506ed772aadc053fabe1a97fd96b26141ce2b3247a5fc97b6f1"
Jul 17 05:16:45.535521 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:16:45.535493    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-4e221674-5ff85f6499-zdth9_odh-ai-gateway-infra(0e7718a9-2705-4130-864b-6d1a9df9c5d6)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9" podUID="0e7718a9-2705-4130-864b-6d1a9df9c5d6"
Jul 17 05:16:47.606640 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:47.606602    2562 generic.go:356] "Generic (PLEG): container finished" podID="c14a1668-297d-4028-96d3-e5be393a0c37" containerID="55b460f7fb559c43cca666b6f5802c26edf366bafa168dab5c2509e6448a6abc" exitCode=0
Jul 17 05:16:47.607202 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:47.606682    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t" event={"ID":"c14a1668-297d-4028-96d3-e5be393a0c37","Type":"ContainerDied","Data":"55b460f7fb559c43cca666b6f5802c26edf366bafa168dab5c2509e6448a6abc"}
Jul 17 05:16:47.607310 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:47.607291    2562 scope.go:117] "RemoveContainer" containerID="55b460f7fb559c43cca666b6f5802c26edf366bafa168dab5c2509e6448a6abc"
Jul 17 05:16:47.608652 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:47.608613    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:16:47.608779 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:47.608726    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:16:47.608779 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:47.608775    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:16:48.613979 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:48.613920    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t" event={"ID":"c14a1668-297d-4028-96d3-e5be393a0c37","Type":"ContainerStarted","Data":"75838fd1bd205c57048ede1d13335cd56ac836d22606f00bf2f78e92ddf4b123"}
Jul 17 05:16:49.139551 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:49.139506    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t"
Jul 17 05:16:49.140567 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:49.140540    2562 patch_prober.go:28] interesting pod/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 17 05:16:49.140673 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:49.140592    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t" podUID="c14a1668-297d-4028-96d3-e5be393a0c37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 17 05:16:50.139884 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:50.139845    2562 patch_prober.go:28] interesting pod/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 17 05:16:50.140317 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:50.139915    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t" podUID="c14a1668-297d-4028-96d3-e5be393a0c37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 17 05:16:50.526398 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:50.526308    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t"
Jul 17 05:16:51.139044 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:51.139011    2562 patch_prober.go:28] interesting pod/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 17 05:16:51.139226 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:51.139074    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t" podUID="c14a1668-297d-4028-96d3-e5be393a0c37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 17 05:16:52.139657 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:52.139621    2562 patch_prober.go:28] interesting pod/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 17 05:16:52.140083 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:52.139690    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t" podUID="c14a1668-297d-4028-96d3-e5be393a0c37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 17 05:16:53.143660 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:53.143624    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t"
Jul 17 05:16:53.145015 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:16:53.144992    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t"
Jul 17 05:17:00.879143 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:17:00.879108    2562 scope.go:117] "RemoveContainer" containerID="0a4ca3bced6f6506ed772aadc053fabe1a97fd96b26141ce2b3247a5fc97b6f1"
Jul 17 05:17:01.678017 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:17:01.677986    2562 generic.go:356] "Generic (PLEG): container finished" podID="0e7718a9-2705-4130-864b-6d1a9df9c5d6" containerID="7cebb055376545ffa2edf63745533edaa24cbd6fae6413900636e138eaab8938" exitCode=1
Jul 17 05:17:01.678232 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:17:01.678029    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9" event={"ID":"0e7718a9-2705-4130-864b-6d1a9df9c5d6","Type":"ContainerDied","Data":"7cebb055376545ffa2edf63745533edaa24cbd6fae6413900636e138eaab8938"}
Jul 17 05:17:01.678232 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:17:01.678079    2562 scope.go:117] "RemoveContainer" containerID="0a4ca3bced6f6506ed772aadc053fabe1a97fd96b26141ce2b3247a5fc97b6f1"
Jul 17 05:17:01.678407 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:17:01.678386    2562 scope.go:117] "RemoveContainer" containerID="7cebb055376545ffa2edf63745533edaa24cbd6fae6413900636e138eaab8938"
Jul 17 05:17:01.678683 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:17:01.678661    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-ait-4e221674-5ff85f6499-zdth9_odh-ai-gateway-infra(0e7718a9-2705-4130-864b-6d1a9df9c5d6)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9" podUID="0e7718a9-2705-4130-864b-6d1a9df9c5d6"
Jul 17 05:17:05.534736 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:17:05.534652    2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9"
Jul 17 05:17:05.535235 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:17:05.535038    2562 scope.go:117] "RemoveContainer" containerID="7cebb055376545ffa2edf63745533edaa24cbd6fae6413900636e138eaab8938"
Jul 17 05:17:05.535279 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:17:05.535244    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-ait-4e221674-5ff85f6499-zdth9_odh-ai-gateway-infra(0e7718a9-2705-4130-864b-6d1a9df9c5d6)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9" podUID="0e7718a9-2705-4130-864b-6d1a9df9c5d6"
Jul 17 05:17:06.532610 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:17:06.532568    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9"
Jul 17 05:17:06.533073 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:17:06.533058    2562 scope.go:117] "RemoveContainer" containerID="7cebb055376545ffa2edf63745533edaa24cbd6fae6413900636e138eaab8938"
Jul 17 05:17:06.533258 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:17:06.533236    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-ait-4e221674-5ff85f6499-zdth9_odh-ai-gateway-infra(0e7718a9-2705-4130-864b-6d1a9df9c5d6)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9" podUID="0e7718a9-2705-4130-864b-6d1a9df9c5d6"
Jul 17 05:17:17.878372 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:17:17.878331    2562 scope.go:117] "RemoveContainer" containerID="7cebb055376545ffa2edf63745533edaa24cbd6fae6413900636e138eaab8938"
Jul 17 05:17:17.878758 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:17:17.878511    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-ait-4e221674-5ff85f6499-zdth9_odh-ai-gateway-infra(0e7718a9-2705-4130-864b-6d1a9df9c5d6)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9" podUID="0e7718a9-2705-4130-864b-6d1a9df9c5d6"
Jul 17 05:17:29.878203 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:17:29.878171    2562 scope.go:117] "RemoveContainer" containerID="7cebb055376545ffa2edf63745533edaa24cbd6fae6413900636e138eaab8938"
Jul 17 05:17:30.826531 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:17:30.826500    2562 generic.go:356] "Generic (PLEG): container finished" podID="0e7718a9-2705-4130-864b-6d1a9df9c5d6" containerID="51560cabe9ad56546fcf7ab9b89d05e2d0d73d148d6b7b0248a4d306945b76ca" exitCode=1
Jul 17 05:17:30.826719 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:17:30.826579    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9" event={"ID":"0e7718a9-2705-4130-864b-6d1a9df9c5d6","Type":"ContainerDied","Data":"51560cabe9ad56546fcf7ab9b89d05e2d0d73d148d6b7b0248a4d306945b76ca"}
Jul 17 05:17:30.826719 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:17:30.826653    2562 scope.go:117] "RemoveContainer" containerID="7cebb055376545ffa2edf63745533edaa24cbd6fae6413900636e138eaab8938"
Jul 17 05:17:30.827070 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:17:30.827056    2562 scope.go:117] "RemoveContainer" containerID="51560cabe9ad56546fcf7ab9b89d05e2d0d73d148d6b7b0248a4d306945b76ca"
Jul 17 05:17:30.827249 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:17:30.827231    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-4e221674-5ff85f6499-zdth9_odh-ai-gateway-infra(0e7718a9-2705-4130-864b-6d1a9df9c5d6)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9" podUID="0e7718a9-2705-4130-864b-6d1a9df9c5d6"
Jul 17 05:17:31.533093 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:17:31.533050    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9"
Jul 17 05:17:31.833591 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:17:31.833560    2562 scope.go:117] "RemoveContainer" containerID="51560cabe9ad56546fcf7ab9b89d05e2d0d73d148d6b7b0248a4d306945b76ca"
Jul 17 05:17:31.833762 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:17:31.833743    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-4e221674-5ff85f6499-zdth9_odh-ai-gateway-infra(0e7718a9-2705-4130-864b-6d1a9df9c5d6)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9" podUID="0e7718a9-2705-4130-864b-6d1a9df9c5d6"
Jul 17 05:17:35.534856 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:17:35.534824    2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9"
Jul 17 05:17:35.535269 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:17:35.535187    2562 scope.go:117] "RemoveContainer" containerID="51560cabe9ad56546fcf7ab9b89d05e2d0d73d148d6b7b0248a4d306945b76ca"
Jul 17 05:17:35.535382 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:17:35.535362    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-4e221674-5ff85f6499-zdth9_odh-ai-gateway-infra(0e7718a9-2705-4130-864b-6d1a9df9c5d6)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9" podUID="0e7718a9-2705-4130-864b-6d1a9df9c5d6"
Jul 17 05:17:49.878235 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:17:49.878201    2562 scope.go:117] "RemoveContainer" containerID="51560cabe9ad56546fcf7ab9b89d05e2d0d73d148d6b7b0248a4d306945b76ca"
Jul 17 05:17:49.878669 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:17:49.878382    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-4e221674-5ff85f6499-zdth9_odh-ai-gateway-infra(0e7718a9-2705-4130-864b-6d1a9df9c5d6)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9" podUID="0e7718a9-2705-4130-864b-6d1a9df9c5d6"
Jul 17 05:18:00.883721 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:18:00.883691    2562 scope.go:117] "RemoveContainer" containerID="51560cabe9ad56546fcf7ab9b89d05e2d0d73d148d6b7b0248a4d306945b76ca"
Jul 17 05:18:00.884106 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:18:00.883907    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-4e221674-5ff85f6499-zdth9_odh-ai-gateway-infra(0e7718a9-2705-4130-864b-6d1a9df9c5d6)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9" podUID="0e7718a9-2705-4130-864b-6d1a9df9c5d6"
Jul 17 05:18:13.877897 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:18:13.877867    2562 scope.go:117] "RemoveContainer" containerID="51560cabe9ad56546fcf7ab9b89d05e2d0d73d148d6b7b0248a4d306945b76ca"
Jul 17 05:18:15.041407 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:18:15.041318    2562 generic.go:356] "Generic (PLEG): container finished" podID="0e7718a9-2705-4130-864b-6d1a9df9c5d6" containerID="c21a0d31ee98d4b0115a58c442e03c0280a5961d9cd1fcc41e27f87e3aee6dde" exitCode=1
Jul 17 05:18:15.041407 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:18:15.041389    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9" event={"ID":"0e7718a9-2705-4130-864b-6d1a9df9c5d6","Type":"ContainerDied","Data":"c21a0d31ee98d4b0115a58c442e03c0280a5961d9cd1fcc41e27f87e3aee6dde"}
Jul 17 05:18:15.042154 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:18:15.041449    2562 scope.go:117] "RemoveContainer" containerID="51560cabe9ad56546fcf7ab9b89d05e2d0d73d148d6b7b0248a4d306945b76ca"
Jul 17 05:18:15.042154 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:18:15.041850    2562 scope.go:117] "RemoveContainer" containerID="c21a0d31ee98d4b0115a58c442e03c0280a5961d9cd1fcc41e27f87e3aee6dde"
Jul 17 05:18:15.042154 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:18:15.042092    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-4e221674-5ff85f6499-zdth9_odh-ai-gateway-infra(0e7718a9-2705-4130-864b-6d1a9df9c5d6)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9" podUID="0e7718a9-2705-4130-864b-6d1a9df9c5d6"
Jul 17 05:18:15.535052 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:18:15.535016    2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9"
Jul 17 05:18:16.047955 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:18:16.047931    2562 scope.go:117] "RemoveContainer" containerID="c21a0d31ee98d4b0115a58c442e03c0280a5961d9cd1fcc41e27f87e3aee6dde"
Jul 17 05:18:16.048336 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:18:16.048124    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-4e221674-5ff85f6499-zdth9_odh-ai-gateway-infra(0e7718a9-2705-4130-864b-6d1a9df9c5d6)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9" podUID="0e7718a9-2705-4130-864b-6d1a9df9c5d6"
Jul 17 05:18:16.532476 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:18:16.532444    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9"
Jul 17 05:18:17.053076 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:18:17.053052    2562 scope.go:117] "RemoveContainer" containerID="c21a0d31ee98d4b0115a58c442e03c0280a5961d9cd1fcc41e27f87e3aee6dde"
Jul 17 05:18:17.053443 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:18:17.053232    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-4e221674-5ff85f6499-zdth9_odh-ai-gateway-infra(0e7718a9-2705-4130-864b-6d1a9df9c5d6)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9" podUID="0e7718a9-2705-4130-864b-6d1a9df9c5d6"
Jul 17 05:18:29.878228 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:18:29.878181    2562 scope.go:117] "RemoveContainer" containerID="c21a0d31ee98d4b0115a58c442e03c0280a5961d9cd1fcc41e27f87e3aee6dde"
Jul 17 05:18:29.878689 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:18:29.878471    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-4e221674-5ff85f6499-zdth9_odh-ai-gateway-infra(0e7718a9-2705-4130-864b-6d1a9df9c5d6)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9" podUID="0e7718a9-2705-4130-864b-6d1a9df9c5d6"
Jul 17 05:18:44.884622 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:18:44.884591    2562 scope.go:117] "RemoveContainer" containerID="c21a0d31ee98d4b0115a58c442e03c0280a5961d9cd1fcc41e27f87e3aee6dde"
Jul 17 05:18:44.885095 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:18:44.884798    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-4e221674-5ff85f6499-zdth9_odh-ai-gateway-infra(0e7718a9-2705-4130-864b-6d1a9df9c5d6)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9" podUID="0e7718a9-2705-4130-864b-6d1a9df9c5d6"
Jul 17 05:18:56.878872 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:18:56.878837    2562 scope.go:117] "RemoveContainer" containerID="c21a0d31ee98d4b0115a58c442e03c0280a5961d9cd1fcc41e27f87e3aee6dde"
Jul 17 05:18:56.879374 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:18:56.879063    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-4e221674-5ff85f6499-zdth9_odh-ai-gateway-infra(0e7718a9-2705-4130-864b-6d1a9df9c5d6)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9" podUID="0e7718a9-2705-4130-864b-6d1a9df9c5d6"
Jul 17 05:19:10.878702 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:10.878671    2562 scope.go:117] "RemoveContainer" containerID="c21a0d31ee98d4b0115a58c442e03c0280a5961d9cd1fcc41e27f87e3aee6dde"
Jul 17 05:19:10.879193 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:19:10.878845    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-4e221674-5ff85f6499-zdth9_odh-ai-gateway-infra(0e7718a9-2705-4130-864b-6d1a9df9c5d6)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9" podUID="0e7718a9-2705-4130-864b-6d1a9df9c5d6"
Jul 17 05:19:23.878894 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:23.878864    2562 scope.go:117] "RemoveContainer" containerID="c21a0d31ee98d4b0115a58c442e03c0280a5961d9cd1fcc41e27f87e3aee6dde"
Jul 17 05:19:23.879331 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:19:23.879096    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-4e221674-5ff85f6499-zdth9_odh-ai-gateway-infra(0e7718a9-2705-4130-864b-6d1a9df9c5d6)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9" podUID="0e7718a9-2705-4130-864b-6d1a9df9c5d6"
Jul 17 05:19:37.878789 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:37.878753    2562 scope.go:117] "RemoveContainer" containerID="c21a0d31ee98d4b0115a58c442e03c0280a5961d9cd1fcc41e27f87e3aee6dde"
Jul 17 05:19:39.474280 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:39.474245    2562 generic.go:356] "Generic (PLEG): container finished" podID="0e7718a9-2705-4130-864b-6d1a9df9c5d6" containerID="989ac1d922c53c9a017f1c0738e701303942d6f25e430f2384e2db3a7a44db31" exitCode=1
Jul 17 05:19:39.474704 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:39.474319    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9" event={"ID":"0e7718a9-2705-4130-864b-6d1a9df9c5d6","Type":"ContainerDied","Data":"989ac1d922c53c9a017f1c0738e701303942d6f25e430f2384e2db3a7a44db31"}
Jul 17 05:19:39.474704 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:39.474384    2562 scope.go:117] "RemoveContainer" containerID="c21a0d31ee98d4b0115a58c442e03c0280a5961d9cd1fcc41e27f87e3aee6dde"
Jul 17 05:19:39.474815 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:39.474712    2562 scope.go:117] "RemoveContainer" containerID="989ac1d922c53c9a017f1c0738e701303942d6f25e430f2384e2db3a7a44db31"
Jul 17 05:19:39.474944 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:19:39.474925    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-4e221674-5ff85f6499-zdth9_odh-ai-gateway-infra(0e7718a9-2705-4130-864b-6d1a9df9c5d6)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9" podUID="0e7718a9-2705-4130-864b-6d1a9df9c5d6"
Jul 17 05:19:40.145774 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:40.145737    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t"]
Jul 17 05:19:40.145990 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:40.145941    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t" podUID="c14a1668-297d-4028-96d3-e5be393a0c37" containerName="istio-proxy" containerID="cri-o://75838fd1bd205c57048ede1d13335cd56ac836d22606f00bf2f78e92ddf4b123" gracePeriod=30
Jul 17 05:19:41.532826 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:41.532791    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9"
Jul 17 05:19:41.533364 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:41.533279    2562 scope.go:117] "RemoveContainer" containerID="989ac1d922c53c9a017f1c0738e701303942d6f25e430f2384e2db3a7a44db31"
Jul 17 05:19:41.533570 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:19:41.533544    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-4e221674-5ff85f6499-zdth9_odh-ai-gateway-infra(0e7718a9-2705-4130-864b-6d1a9df9c5d6)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9" podUID="0e7718a9-2705-4130-864b-6d1a9df9c5d6"
Jul 17 05:19:45.400040 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:45.400014    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t"
Jul 17 05:19:45.508282 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:45.508247    2562 generic.go:356] "Generic (PLEG): container finished" podID="c14a1668-297d-4028-96d3-e5be393a0c37" containerID="75838fd1bd205c57048ede1d13335cd56ac836d22606f00bf2f78e92ddf4b123" exitCode=0
Jul 17 05:19:45.508479 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:45.508307    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t"
Jul 17 05:19:45.508479 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:45.508337    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t" event={"ID":"c14a1668-297d-4028-96d3-e5be393a0c37","Type":"ContainerDied","Data":"75838fd1bd205c57048ede1d13335cd56ac836d22606f00bf2f78e92ddf4b123"}
Jul 17 05:19:45.508479 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:45.508393    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t" event={"ID":"c14a1668-297d-4028-96d3-e5be393a0c37","Type":"ContainerDied","Data":"190c051664d944f279b7a0beabad964e215a29636b061e48d199e1a577873bda"}
Jul 17 05:19:45.508479 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:45.508417    2562 scope.go:117] "RemoveContainer" containerID="75838fd1bd205c57048ede1d13335cd56ac836d22606f00bf2f78e92ddf4b123"
Jul 17 05:19:45.510571 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:45.510551    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c14a1668-297d-4028-96d3-e5be393a0c37-credential-socket\") pod \"c14a1668-297d-4028-96d3-e5be393a0c37\" (UID: \"c14a1668-297d-4028-96d3-e5be393a0c37\") "
Jul 17 05:19:45.510686 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:45.510594    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c14a1668-297d-4028-96d3-e5be393a0c37-istio-token\") pod \"c14a1668-297d-4028-96d3-e5be393a0c37\" (UID: \"c14a1668-297d-4028-96d3-e5be393a0c37\") "
Jul 17 05:19:45.510686 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:45.510614    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c14a1668-297d-4028-96d3-e5be393a0c37-istio-podinfo\") pod \"c14a1668-297d-4028-96d3-e5be393a0c37\" (UID: \"c14a1668-297d-4028-96d3-e5be393a0c37\") "
Jul 17 05:19:45.510879 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:45.510689    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m7j9s\" (UniqueName: \"kubernetes.io/projected/c14a1668-297d-4028-96d3-e5be393a0c37-kube-api-access-m7j9s\") pod \"c14a1668-297d-4028-96d3-e5be393a0c37\" (UID: \"c14a1668-297d-4028-96d3-e5be393a0c37\") "
Jul 17 05:19:45.510879 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:45.510715    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c14a1668-297d-4028-96d3-e5be393a0c37-istio-data\") pod \"c14a1668-297d-4028-96d3-e5be393a0c37\" (UID: \"c14a1668-297d-4028-96d3-e5be393a0c37\") "
Jul 17 05:19:45.510879 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:45.510788    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c14a1668-297d-4028-96d3-e5be393a0c37-workload-certs\") pod \"c14a1668-297d-4028-96d3-e5be393a0c37\" (UID: \"c14a1668-297d-4028-96d3-e5be393a0c37\") "
Jul 17 05:19:45.510879 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:45.510816    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c14a1668-297d-4028-96d3-e5be393a0c37-istiod-ca-cert\") pod \"c14a1668-297d-4028-96d3-e5be393a0c37\" (UID: \"c14a1668-297d-4028-96d3-e5be393a0c37\") "
Jul 17 05:19:45.510879 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:45.510816    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c14a1668-297d-4028-96d3-e5be393a0c37-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c14a1668-297d-4028-96d3-e5be393a0c37" (UID: "c14a1668-297d-4028-96d3-e5be393a0c37"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:19:45.510879 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:45.510845    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c14a1668-297d-4028-96d3-e5be393a0c37-istio-envoy\") pod \"c14a1668-297d-4028-96d3-e5be393a0c37\" (UID: \"c14a1668-297d-4028-96d3-e5be393a0c37\") "
Jul 17 05:19:45.511201 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:45.510893    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c14a1668-297d-4028-96d3-e5be393a0c37-workload-socket\") pod \"c14a1668-297d-4028-96d3-e5be393a0c37\" (UID: \"c14a1668-297d-4028-96d3-e5be393a0c37\") "
Jul 17 05:19:45.511201 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:45.511063    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c14a1668-297d-4028-96d3-e5be393a0c37-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c14a1668-297d-4028-96d3-e5be393a0c37" (UID: "c14a1668-297d-4028-96d3-e5be393a0c37"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:19:45.511301 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:45.511276    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c14a1668-297d-4028-96d3-e5be393a0c37-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c14a1668-297d-4028-96d3-e5be393a0c37" (UID: "c14a1668-297d-4028-96d3-e5be393a0c37"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:19:45.511301 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:45.511283    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c14a1668-297d-4028-96d3-e5be393a0c37-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c14a1668-297d-4028-96d3-e5be393a0c37" (UID: "c14a1668-297d-4028-96d3-e5be393a0c37"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:19:45.511412 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:45.511341    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c14a1668-297d-4028-96d3-e5be393a0c37-workload-certs\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:19:45.511412 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:45.511364    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c14a1668-297d-4028-96d3-e5be393a0c37-credential-socket\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:19:45.511913 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:45.511883    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c14a1668-297d-4028-96d3-e5be393a0c37-istio-data" (OuterVolumeSpecName: "istio-data") pod "c14a1668-297d-4028-96d3-e5be393a0c37" (UID: "c14a1668-297d-4028-96d3-e5be393a0c37"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:19:45.513323 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:45.513293    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c14a1668-297d-4028-96d3-e5be393a0c37-istio-token" (OuterVolumeSpecName: "istio-token") pod "c14a1668-297d-4028-96d3-e5be393a0c37" (UID: "c14a1668-297d-4028-96d3-e5be393a0c37"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:19:45.513451 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:45.513436    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c14a1668-297d-4028-96d3-e5be393a0c37-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c14a1668-297d-4028-96d3-e5be393a0c37" (UID: "c14a1668-297d-4028-96d3-e5be393a0c37"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 05:19:45.513517 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:45.513453    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c14a1668-297d-4028-96d3-e5be393a0c37-kube-api-access-m7j9s" (OuterVolumeSpecName: "kube-api-access-m7j9s") pod "c14a1668-297d-4028-96d3-e5be393a0c37" (UID: "c14a1668-297d-4028-96d3-e5be393a0c37"). InnerVolumeSpecName "kube-api-access-m7j9s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:19:45.513628 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:45.513603    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c14a1668-297d-4028-96d3-e5be393a0c37-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c14a1668-297d-4028-96d3-e5be393a0c37" (UID: "c14a1668-297d-4028-96d3-e5be393a0c37"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:19:45.532789 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:45.532764    2562 scope.go:117] "RemoveContainer" containerID="55b460f7fb559c43cca666b6f5802c26edf366bafa168dab5c2509e6448a6abc"
Jul 17 05:19:45.534851 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:45.534830    2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9"
Jul 17 05:19:45.535536 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:45.535520    2562 scope.go:117] "RemoveContainer" containerID="989ac1d922c53c9a017f1c0738e701303942d6f25e430f2384e2db3a7a44db31"
Jul 17 05:19:45.535755 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:19:45.535734    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-4e221674-5ff85f6499-zdth9_odh-ai-gateway-infra(0e7718a9-2705-4130-864b-6d1a9df9c5d6)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9" podUID="0e7718a9-2705-4130-864b-6d1a9df9c5d6"
Jul 17 05:19:45.541164 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:45.541147    2562 scope.go:117] "RemoveContainer" containerID="75838fd1bd205c57048ede1d13335cd56ac836d22606f00bf2f78e92ddf4b123"
Jul 17 05:19:45.541482 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:19:45.541459    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75838fd1bd205c57048ede1d13335cd56ac836d22606f00bf2f78e92ddf4b123\": container with ID starting with 75838fd1bd205c57048ede1d13335cd56ac836d22606f00bf2f78e92ddf4b123 not found: ID does not exist" containerID="75838fd1bd205c57048ede1d13335cd56ac836d22606f00bf2f78e92ddf4b123"
Jul 17 05:19:45.541577 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:45.541487    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75838fd1bd205c57048ede1d13335cd56ac836d22606f00bf2f78e92ddf4b123"} err="failed to get container status \"75838fd1bd205c57048ede1d13335cd56ac836d22606f00bf2f78e92ddf4b123\": rpc error: code = NotFound desc = could not find container \"75838fd1bd205c57048ede1d13335cd56ac836d22606f00bf2f78e92ddf4b123\": container with ID starting with 75838fd1bd205c57048ede1d13335cd56ac836d22606f00bf2f78e92ddf4b123 not found: ID does not exist"
Jul 17 05:19:45.541577 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:45.541504    2562 scope.go:117] "RemoveContainer" containerID="55b460f7fb559c43cca666b6f5802c26edf366bafa168dab5c2509e6448a6abc"
Jul 17 05:19:45.541737 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:19:45.541718    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55b460f7fb559c43cca666b6f5802c26edf366bafa168dab5c2509e6448a6abc\": container with ID starting with 55b460f7fb559c43cca666b6f5802c26edf366bafa168dab5c2509e6448a6abc not found: ID does not exist" containerID="55b460f7fb559c43cca666b6f5802c26edf366bafa168dab5c2509e6448a6abc"
Jul 17 05:19:45.541778 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:45.541745    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55b460f7fb559c43cca666b6f5802c26edf366bafa168dab5c2509e6448a6abc"} err="failed to get container status \"55b460f7fb559c43cca666b6f5802c26edf366bafa168dab5c2509e6448a6abc\": rpc error: code = NotFound desc = could not find container \"55b460f7fb559c43cca666b6f5802c26edf366bafa168dab5c2509e6448a6abc\": container with ID starting with 55b460f7fb559c43cca666b6f5802c26edf366bafa168dab5c2509e6448a6abc not found: ID does not exist"
Jul 17 05:19:45.612203 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:45.612165    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m7j9s\" (UniqueName: \"kubernetes.io/projected/c14a1668-297d-4028-96d3-e5be393a0c37-kube-api-access-m7j9s\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:19:45.612203 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:45.612193    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c14a1668-297d-4028-96d3-e5be393a0c37-istio-data\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:19:45.612203 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:45.612203    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c14a1668-297d-4028-96d3-e5be393a0c37-istiod-ca-cert\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:19:45.612203 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:45.612211    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c14a1668-297d-4028-96d3-e5be393a0c37-istio-envoy\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:19:45.612203 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:45.612219    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c14a1668-297d-4028-96d3-e5be393a0c37-workload-socket\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:19:45.612543 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:45.612228    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c14a1668-297d-4028-96d3-e5be393a0c37-istio-token\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:19:45.612543 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:45.612244    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c14a1668-297d-4028-96d3-e5be393a0c37-istio-podinfo\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:19:45.835187 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:45.835147    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t"]
Jul 17 05:19:45.840935 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:45.840903    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-4e221674-openshift-default-69c686f59c-9nz7t"]
Jul 17 05:19:45.992234 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:45.992200    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9"]
Jul 17 05:19:46.652333 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:46.652298    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9"
Jul 17 05:19:46.726529 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:46.726496    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e7718a9-2705-4130-864b-6d1a9df9c5d6-maas-api-tls\") pod \"0e7718a9-2705-4130-864b-6d1a9df9c5d6\" (UID: \"0e7718a9-2705-4130-864b-6d1a9df9c5d6\") "
Jul 17 05:19:46.726714 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:46.726546    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rwd6t\" (UniqueName: \"kubernetes.io/projected/0e7718a9-2705-4130-864b-6d1a9df9c5d6-kube-api-access-rwd6t\") pod \"0e7718a9-2705-4130-864b-6d1a9df9c5d6\" (UID: \"0e7718a9-2705-4130-864b-6d1a9df9c5d6\") "
Jul 17 05:19:46.726714 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:46.726571    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e7718a9-2705-4130-864b-6d1a9df9c5d6-openshift-service-ca\") pod \"0e7718a9-2705-4130-864b-6d1a9df9c5d6\" (UID: \"0e7718a9-2705-4130-864b-6d1a9df9c5d6\") "
Jul 17 05:19:46.727031 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:46.727010    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e7718a9-2705-4130-864b-6d1a9df9c5d6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0e7718a9-2705-4130-864b-6d1a9df9c5d6" (UID: "0e7718a9-2705-4130-864b-6d1a9df9c5d6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:19:46.728983 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:46.728926    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e7718a9-2705-4130-864b-6d1a9df9c5d6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0e7718a9-2705-4130-864b-6d1a9df9c5d6" (UID: "0e7718a9-2705-4130-864b-6d1a9df9c5d6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 05:19:46.728983 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:46.728930    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e7718a9-2705-4130-864b-6d1a9df9c5d6-kube-api-access-rwd6t" (OuterVolumeSpecName: "kube-api-access-rwd6t") pod "0e7718a9-2705-4130-864b-6d1a9df9c5d6" (UID: "0e7718a9-2705-4130-864b-6d1a9df9c5d6"). InnerVolumeSpecName "kube-api-access-rwd6t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:19:46.828033 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:46.828000    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e7718a9-2705-4130-864b-6d1a9df9c5d6-maas-api-tls\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:19:46.828033 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:46.828030    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rwd6t\" (UniqueName: \"kubernetes.io/projected/0e7718a9-2705-4130-864b-6d1a9df9c5d6-kube-api-access-rwd6t\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:19:46.828033 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:46.828039    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e7718a9-2705-4130-864b-6d1a9df9c5d6-openshift-service-ca\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:19:46.884475 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:46.884444    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c14a1668-297d-4028-96d3-e5be393a0c37" path="/var/lib/kubelet/pods/c14a1668-297d-4028-96d3-e5be393a0c37/volumes"
Jul 17 05:19:47.519897 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:47.519861    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9"
Jul 17 05:19:47.520080 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:47.519911    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9" event={"ID":"0e7718a9-2705-4130-864b-6d1a9df9c5d6","Type":"ContainerDied","Data":"84bfc675cac0c8ff7755259ca24c7ec8aab745a6e2d0fdf1421f00189355e93b"}
Jul 17 05:19:47.520080 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:47.519990    2562 scope.go:117] "RemoveContainer" containerID="989ac1d922c53c9a017f1c0738e701303942d6f25e430f2384e2db3a7a44db31"
Jul 17 05:19:47.538124 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:47.538085    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9"]
Jul 17 05:19:47.542458 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:47.542432    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4e221674-5ff85f6499-zdth9"]
Jul 17 05:19:48.884286 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:48.884250    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e7718a9-2705-4130-864b-6d1a9df9c5d6" path="/var/lib/kubelet/pods/0e7718a9-2705-4130-864b-6d1a9df9c5d6/volumes"
Jul 17 05:19:52.510654 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.510619    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl"]
Jul 17 05:19:52.511310 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.511291    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e7718a9-2705-4130-864b-6d1a9df9c5d6" containerName="maas-api"
Jul 17 05:19:52.511380 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.511313    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e7718a9-2705-4130-864b-6d1a9df9c5d6" containerName="maas-api"
Jul 17 05:19:52.511380 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.511329    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e7718a9-2705-4130-864b-6d1a9df9c5d6" containerName="maas-api"
Jul 17 05:19:52.511380 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.511338    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e7718a9-2705-4130-864b-6d1a9df9c5d6" containerName="maas-api"
Jul 17 05:19:52.511380 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.511357    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c14a1668-297d-4028-96d3-e5be393a0c37" containerName="istio-proxy"
Jul 17 05:19:52.511380 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.511366    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c14a1668-297d-4028-96d3-e5be393a0c37" containerName="istio-proxy"
Jul 17 05:19:52.511380 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.511377    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c14a1668-297d-4028-96d3-e5be393a0c37" containerName="istio-proxy"
Jul 17 05:19:52.511380 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.511385    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c14a1668-297d-4028-96d3-e5be393a0c37" containerName="istio-proxy"
Jul 17 05:19:52.511720 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.511399    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e7718a9-2705-4130-864b-6d1a9df9c5d6" containerName="maas-api"
Jul 17 05:19:52.511720 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.511407    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e7718a9-2705-4130-864b-6d1a9df9c5d6" containerName="maas-api"
Jul 17 05:19:52.511720 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.511506    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e7718a9-2705-4130-864b-6d1a9df9c5d6" containerName="maas-api"
Jul 17 05:19:52.511720 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.511518    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e7718a9-2705-4130-864b-6d1a9df9c5d6" containerName="maas-api"
Jul 17 05:19:52.511720 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.511526    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e7718a9-2705-4130-864b-6d1a9df9c5d6" containerName="maas-api"
Jul 17 05:19:52.511720 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.511535    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c14a1668-297d-4028-96d3-e5be393a0c37" containerName="istio-proxy"
Jul 17 05:19:52.511720 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.511547    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c14a1668-297d-4028-96d3-e5be393a0c37" containerName="istio-proxy"
Jul 17 05:19:52.511720 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.511557    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e7718a9-2705-4130-864b-6d1a9df9c5d6" containerName="maas-api"
Jul 17 05:19:52.516455 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.516436    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl"
Jul 17 05:19:52.519004 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.518956    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-c8a383ce-gw-openshift-default-dockercfg-9xfhd\""
Jul 17 05:19:52.524754 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.524728    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl"]
Jul 17 05:19:52.579650 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.579616    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e823110b-3d4e-48bf-a436-5a219307d70e-workload-certs\") pod \"e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl\" (UID: \"e823110b-3d4e-48bf-a436-5a219307d70e\") " pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl"
Jul 17 05:19:52.579841 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.579673    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e823110b-3d4e-48bf-a436-5a219307d70e-istio-podinfo\") pod \"e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl\" (UID: \"e823110b-3d4e-48bf-a436-5a219307d70e\") " pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl"
Jul 17 05:19:52.579841 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.579699    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e823110b-3d4e-48bf-a436-5a219307d70e-credential-socket\") pod \"e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl\" (UID: \"e823110b-3d4e-48bf-a436-5a219307d70e\") " pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl"
Jul 17 05:19:52.579841 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.579730    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dk5qs\" (UniqueName: \"kubernetes.io/projected/e823110b-3d4e-48bf-a436-5a219307d70e-kube-api-access-dk5qs\") pod \"e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl\" (UID: \"e823110b-3d4e-48bf-a436-5a219307d70e\") " pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl"
Jul 17 05:19:52.579841 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.579763    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e823110b-3d4e-48bf-a436-5a219307d70e-istiod-ca-cert\") pod \"e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl\" (UID: \"e823110b-3d4e-48bf-a436-5a219307d70e\") " pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl"
Jul 17 05:19:52.579841 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.579795    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e823110b-3d4e-48bf-a436-5a219307d70e-istio-envoy\") pod \"e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl\" (UID: \"e823110b-3d4e-48bf-a436-5a219307d70e\") " pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl"
Jul 17 05:19:52.579841 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.579812    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e823110b-3d4e-48bf-a436-5a219307d70e-workload-socket\") pod \"e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl\" (UID: \"e823110b-3d4e-48bf-a436-5a219307d70e\") " pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl"
Jul 17 05:19:52.580080 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.579882    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e823110b-3d4e-48bf-a436-5a219307d70e-istio-data\") pod \"e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl\" (UID: \"e823110b-3d4e-48bf-a436-5a219307d70e\") " pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl"
Jul 17 05:19:52.580080 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.579919    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e823110b-3d4e-48bf-a436-5a219307d70e-istio-token\") pod \"e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl\" (UID: \"e823110b-3d4e-48bf-a436-5a219307d70e\") " pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl"
Jul 17 05:19:52.680577 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.680538    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dk5qs\" (UniqueName: \"kubernetes.io/projected/e823110b-3d4e-48bf-a436-5a219307d70e-kube-api-access-dk5qs\") pod \"e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl\" (UID: \"e823110b-3d4e-48bf-a436-5a219307d70e\") " pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl"
Jul 17 05:19:52.680577 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.680582    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e823110b-3d4e-48bf-a436-5a219307d70e-istiod-ca-cert\") pod \"e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl\" (UID: \"e823110b-3d4e-48bf-a436-5a219307d70e\") " pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl"
Jul 17 05:19:52.680820 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.680709    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e823110b-3d4e-48bf-a436-5a219307d70e-istio-envoy\") pod \"e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl\" (UID: \"e823110b-3d4e-48bf-a436-5a219307d70e\") " pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl"
Jul 17 05:19:52.680820 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.680742    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e823110b-3d4e-48bf-a436-5a219307d70e-workload-socket\") pod \"e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl\" (UID: \"e823110b-3d4e-48bf-a436-5a219307d70e\") " pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl"
Jul 17 05:19:52.680820 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.680817    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e823110b-3d4e-48bf-a436-5a219307d70e-istio-data\") pod \"e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl\" (UID: \"e823110b-3d4e-48bf-a436-5a219307d70e\") " pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl"
Jul 17 05:19:52.681009 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.680875    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e823110b-3d4e-48bf-a436-5a219307d70e-istio-token\") pod \"e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl\" (UID: \"e823110b-3d4e-48bf-a436-5a219307d70e\") " pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl"
Jul 17 05:19:52.681009 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.680928    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e823110b-3d4e-48bf-a436-5a219307d70e-workload-certs\") pod \"e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl\" (UID: \"e823110b-3d4e-48bf-a436-5a219307d70e\") " pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl"
Jul 17 05:19:52.681009 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.680997    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e823110b-3d4e-48bf-a436-5a219307d70e-istio-podinfo\") pod \"e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl\" (UID: \"e823110b-3d4e-48bf-a436-5a219307d70e\") " pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl"
Jul 17 05:19:52.681172 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.681027    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e823110b-3d4e-48bf-a436-5a219307d70e-credential-socket\") pod \"e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl\" (UID: \"e823110b-3d4e-48bf-a436-5a219307d70e\") " pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl"
Jul 17 05:19:52.681446 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.681282    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e823110b-3d4e-48bf-a436-5a219307d70e-workload-certs\") pod \"e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl\" (UID: \"e823110b-3d4e-48bf-a436-5a219307d70e\") " pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl"
Jul 17 05:19:52.681446 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.681314    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e823110b-3d4e-48bf-a436-5a219307d70e-istio-data\") pod \"e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl\" (UID: \"e823110b-3d4e-48bf-a436-5a219307d70e\") " pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl"
Jul 17 05:19:52.681446 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.681369    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e823110b-3d4e-48bf-a436-5a219307d70e-istiod-ca-cert\") pod \"e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl\" (UID: \"e823110b-3d4e-48bf-a436-5a219307d70e\") " pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl"
Jul 17 05:19:52.681446 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.681421    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e823110b-3d4e-48bf-a436-5a219307d70e-credential-socket\") pod \"e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl\" (UID: \"e823110b-3d4e-48bf-a436-5a219307d70e\") " pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl"
Jul 17 05:19:52.681992 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.681503    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e823110b-3d4e-48bf-a436-5a219307d70e-workload-socket\") pod \"e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl\" (UID: \"e823110b-3d4e-48bf-a436-5a219307d70e\") " pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl"
Jul 17 05:19:52.683321 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.683296    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e823110b-3d4e-48bf-a436-5a219307d70e-istio-envoy\") pod \"e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl\" (UID: \"e823110b-3d4e-48bf-a436-5a219307d70e\") " pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl"
Jul 17 05:19:52.683665 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.683647    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e823110b-3d4e-48bf-a436-5a219307d70e-istio-podinfo\") pod \"e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl\" (UID: \"e823110b-3d4e-48bf-a436-5a219307d70e\") " pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl"
Jul 17 05:19:52.692467 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.692443    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dk5qs\" (UniqueName: \"kubernetes.io/projected/e823110b-3d4e-48bf-a436-5a219307d70e-kube-api-access-dk5qs\") pod \"e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl\" (UID: \"e823110b-3d4e-48bf-a436-5a219307d70e\") " pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl"
Jul 17 05:19:52.692593 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.692482    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e823110b-3d4e-48bf-a436-5a219307d70e-istio-token\") pod \"e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl\" (UID: \"e823110b-3d4e-48bf-a436-5a219307d70e\") " pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl"
Jul 17 05:19:52.830241 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.830202    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl"
Jul 17 05:19:52.973841 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.973807    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl"]
Jul 17 05:19:52.975869 ip-10-0-132-19 kubenswrapper[2562]: W0717 05:19:52.975838    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode823110b_3d4e_48bf_a436_5a219307d70e.slice/crio-222e3052b7df2bc1c74b241bb3b26ee029e3b5dc873e3aa729ad850a9341a035 WatchSource:0}: Error finding container 222e3052b7df2bc1c74b241bb3b26ee029e3b5dc873e3aa729ad850a9341a035: Status 404 returned error can't find the container with id 222e3052b7df2bc1c74b241bb3b26ee029e3b5dc873e3aa729ad850a9341a035
Jul 17 05:19:52.978312 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.978271    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:19:52.978406 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.978344    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:19:52.978406 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:52.978373    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:19:53.554309 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:53.554268    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl" event={"ID":"e823110b-3d4e-48bf-a436-5a219307d70e","Type":"ContainerStarted","Data":"c4ce9b45312b8a605d5a6163bb8469b43cc60859f5bc4a366fdc482995545221"}
Jul 17 05:19:53.554688 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:53.554335    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl" event={"ID":"e823110b-3d4e-48bf-a436-5a219307d70e","Type":"ContainerStarted","Data":"222e3052b7df2bc1c74b241bb3b26ee029e3b5dc873e3aa729ad850a9341a035"}
Jul 17 05:19:53.575308 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:53.575254    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl" podStartSLOduration=1.575238052 podStartE2EDuration="1.575238052s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:19:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:19:53.573310795 +0000 UTC m=+1781.256904478" watchObservedRunningTime="2026-07-17 05:19:53.575238052 +0000 UTC m=+1781.258831708"
Jul 17 05:19:53.830773 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:53.830724    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl"
Jul 17 05:19:54.102997 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:54.102905    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck"]
Jul 17 05:19:54.103349 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:54.103332    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e7718a9-2705-4130-864b-6d1a9df9c5d6" containerName="maas-api"
Jul 17 05:19:54.103429 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:54.103353    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e7718a9-2705-4130-864b-6d1a9df9c5d6" containerName="maas-api"
Jul 17 05:19:54.103429 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:54.103380    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e7718a9-2705-4130-864b-6d1a9df9c5d6" containerName="maas-api"
Jul 17 05:19:54.103429 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:54.103386    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e7718a9-2705-4130-864b-6d1a9df9c5d6" containerName="maas-api"
Jul 17 05:19:54.103582 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:54.103454    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e7718a9-2705-4130-864b-6d1a9df9c5d6" containerName="maas-api"
Jul 17 05:19:54.103582 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:54.103466    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e7718a9-2705-4130-864b-6d1a9df9c5d6" containerName="maas-api"
Jul 17 05:19:54.106153 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:54.106120    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck"
Jul 17 05:19:54.109020 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:54.108994    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-xwcd7\""
Jul 17 05:19:54.109156 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:54.108992    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-c8a383ce-serving-cert\""
Jul 17 05:19:54.116836 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:54.116807    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck"]
Jul 17 05:19:54.195086 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:54.195050    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/370367ab-def2-4733-85db-8983d56c7c96-openshift-service-ca\") pod \"maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck\" (UID: \"370367ab-def2-4733-85db-8983d56c7c96\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck"
Jul 17 05:19:54.195273 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:54.195117    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/370367ab-def2-4733-85db-8983d56c7c96-maas-api-tls\") pod \"maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck\" (UID: \"370367ab-def2-4733-85db-8983d56c7c96\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck"
Jul 17 05:19:54.195273 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:54.195201    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcvfw\" (UniqueName: \"kubernetes.io/projected/370367ab-def2-4733-85db-8983d56c7c96-kube-api-access-zcvfw\") pod \"maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck\" (UID: \"370367ab-def2-4733-85db-8983d56c7c96\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck"
Jul 17 05:19:54.296123 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:54.296079    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/370367ab-def2-4733-85db-8983d56c7c96-maas-api-tls\") pod \"maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck\" (UID: \"370367ab-def2-4733-85db-8983d56c7c96\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck"
Jul 17 05:19:54.296302 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:54.296166    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcvfw\" (UniqueName: \"kubernetes.io/projected/370367ab-def2-4733-85db-8983d56c7c96-kube-api-access-zcvfw\") pod \"maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck\" (UID: \"370367ab-def2-4733-85db-8983d56c7c96\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck"
Jul 17 05:19:54.296302 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:54.296278    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/370367ab-def2-4733-85db-8983d56c7c96-openshift-service-ca\") pod \"maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck\" (UID: \"370367ab-def2-4733-85db-8983d56c7c96\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck"
Jul 17 05:19:54.296854 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:54.296828    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/370367ab-def2-4733-85db-8983d56c7c96-openshift-service-ca\") pod \"maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck\" (UID: \"370367ab-def2-4733-85db-8983d56c7c96\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck"
Jul 17 05:19:54.298729 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:54.298705    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/370367ab-def2-4733-85db-8983d56c7c96-maas-api-tls\") pod \"maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck\" (UID: \"370367ab-def2-4733-85db-8983d56c7c96\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck"
Jul 17 05:19:54.304171 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:54.304147    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcvfw\" (UniqueName: \"kubernetes.io/projected/370367ab-def2-4733-85db-8983d56c7c96-kube-api-access-zcvfw\") pod \"maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck\" (UID: \"370367ab-def2-4733-85db-8983d56c7c96\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck"
Jul 17 05:19:54.419575 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:54.419486    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck"
Jul 17 05:19:54.565432 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:54.565388    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck"]
Jul 17 05:19:54.571875 ip-10-0-132-19 kubenswrapper[2562]: W0717 05:19:54.571832    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod370367ab_def2_4733_85db_8983d56c7c96.slice/crio-bf2671f53cc2101fa01aaa033e5a635bd93c5c7083a68eda3435906f18ff516d WatchSource:0}: Error finding container bf2671f53cc2101fa01aaa033e5a635bd93c5c7083a68eda3435906f18ff516d: Status 404 returned error can't find the container with id bf2671f53cc2101fa01aaa033e5a635bd93c5c7083a68eda3435906f18ff516d
Jul 17 05:19:54.836309 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:54.836278    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl"
Jul 17 05:19:55.565802 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:55.565766    2562 generic.go:356] "Generic (PLEG): container finished" podID="370367ab-def2-4733-85db-8983d56c7c96" containerID="38ed5df661f2d635a1f04ed85b74c286786b60fc78b2b25260285868ddf57e26" exitCode=1
Jul 17 05:19:55.566304 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:55.565841    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck" event={"ID":"370367ab-def2-4733-85db-8983d56c7c96","Type":"ContainerDied","Data":"38ed5df661f2d635a1f04ed85b74c286786b60fc78b2b25260285868ddf57e26"}
Jul 17 05:19:55.566304 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:55.565878    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck" event={"ID":"370367ab-def2-4733-85db-8983d56c7c96","Type":"ContainerStarted","Data":"bf2671f53cc2101fa01aaa033e5a635bd93c5c7083a68eda3435906f18ff516d"}
Jul 17 05:19:55.566304 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:55.566133    2562 scope.go:117] "RemoveContainer" containerID="38ed5df661f2d635a1f04ed85b74c286786b60fc78b2b25260285868ddf57e26"
Jul 17 05:19:55.566304 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:55.566243    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl"
Jul 17 05:19:55.567257 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:55.567239    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl"
Jul 17 05:19:56.572833 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:56.572801    2562 generic.go:356] "Generic (PLEG): container finished" podID="370367ab-def2-4733-85db-8983d56c7c96" containerID="fe51587690dd68c6b74e959e0e988c1597fe227d500a3ceb816b0d12d51170ce" exitCode=1
Jul 17 05:19:56.573306 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:56.572886    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck" event={"ID":"370367ab-def2-4733-85db-8983d56c7c96","Type":"ContainerDied","Data":"fe51587690dd68c6b74e959e0e988c1597fe227d500a3ceb816b0d12d51170ce"}
Jul 17 05:19:56.573306 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:56.572946    2562 scope.go:117] "RemoveContainer" containerID="38ed5df661f2d635a1f04ed85b74c286786b60fc78b2b25260285868ddf57e26"
Jul 17 05:19:56.573306 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:56.573248    2562 scope.go:117] "RemoveContainer" containerID="fe51587690dd68c6b74e959e0e988c1597fe227d500a3ceb816b0d12d51170ce"
Jul 17 05:19:56.573505 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:19:56.573458    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck_odh-ai-gateway-infra(370367ab-def2-4733-85db-8983d56c7c96)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck" podUID="370367ab-def2-4733-85db-8983d56c7c96"
Jul 17 05:19:57.579876 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:57.579849    2562 scope.go:117] "RemoveContainer" containerID="fe51587690dd68c6b74e959e0e988c1597fe227d500a3ceb816b0d12d51170ce"
Jul 17 05:19:57.580282 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:19:57.580062    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck_odh-ai-gateway-infra(370367ab-def2-4733-85db-8983d56c7c96)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck" podUID="370367ab-def2-4733-85db-8983d56c7c96"
Jul 17 05:19:59.419793 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:59.419756    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck"
Jul 17 05:19:59.420238 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:19:59.420212    2562 scope.go:117] "RemoveContainer" containerID="fe51587690dd68c6b74e959e0e988c1597fe227d500a3ceb816b0d12d51170ce"
Jul 17 05:19:59.420409 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:19:59.420389    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck_odh-ai-gateway-infra(370367ab-def2-4733-85db-8983d56c7c96)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck" podUID="370367ab-def2-4733-85db-8983d56c7c96"
Jul 17 05:20:04.420671 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:20:04.420577    2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck"
Jul 17 05:20:04.421163 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:20:04.421038    2562 scope.go:117] "RemoveContainer" containerID="fe51587690dd68c6b74e959e0e988c1597fe227d500a3ceb816b0d12d51170ce"
Jul 17 05:20:04.421251 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:20:04.421228    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck_odh-ai-gateway-infra(370367ab-def2-4733-85db-8983d56c7c96)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck" podUID="370367ab-def2-4733-85db-8983d56c7c96"
Jul 17 05:20:05.621452 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:20:05.621420    2562 generic.go:356] "Generic (PLEG): container finished" podID="e823110b-3d4e-48bf-a436-5a219307d70e" containerID="c4ce9b45312b8a605d5a6163bb8469b43cc60859f5bc4a366fdc482995545221" exitCode=0
Jul 17 05:20:05.621952 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:20:05.621497    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl" event={"ID":"e823110b-3d4e-48bf-a436-5a219307d70e","Type":"ContainerDied","Data":"c4ce9b45312b8a605d5a6163bb8469b43cc60859f5bc4a366fdc482995545221"}
Jul 17 05:20:05.621952 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:20:05.621898    2562 scope.go:117] "RemoveContainer" containerID="c4ce9b45312b8a605d5a6163bb8469b43cc60859f5bc4a366fdc482995545221"
Jul 17 05:20:05.622989 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:20:05.622946    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:20:05.623062 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:20:05.623027    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:20:05.623101 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:20:05.623089    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:20:06.630100 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:20:06.630063    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl" event={"ID":"e823110b-3d4e-48bf-a436-5a219307d70e","Type":"ContainerStarted","Data":"c30d19dcede625fd15e6da38cf441d43a205ba8e2fc182fcf32768fb40f32dbe"}
Jul 17 05:20:06.831296 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:20:06.831255    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl"
Jul 17 05:20:06.832560 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:20:06.832534    2562 patch_prober.go:28] interesting pod/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Jul 17 05:20:06.832676 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:20:06.832611    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl" podUID="e823110b-3d4e-48bf-a436-5a219307d70e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Jul 17 05:20:07.830809 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:20:07.830777    2562 patch_prober.go:28] interesting pod/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Jul 17 05:20:07.831241 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:20:07.830835    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl" podUID="e823110b-3d4e-48bf-a436-5a219307d70e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Jul 17 05:20:08.830825 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:20:08.830791    2562 patch_prober.go:28] interesting pod/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Jul 17 05:20:08.831257 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:20:08.830846    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl" podUID="e823110b-3d4e-48bf-a436-5a219307d70e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Jul 17 05:20:09.830653 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:20:09.830617    2562 patch_prober.go:28] interesting pod/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Jul 17 05:20:09.830840 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:20:09.830703    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl" podUID="e823110b-3d4e-48bf-a436-5a219307d70e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Jul 17 05:20:10.566444 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:20:10.566400    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl"
Jul 17 05:20:10.835523 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:20:10.835489    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl"
Jul 17 05:20:10.842322 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:20:10.842291    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl"
Jul 17 05:20:12.891952 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:20:12.891929    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tx9gj_c1baeee2-350c-43c5-9a04-fb34378edb5c/ovn-acl-logging/0.log"
Jul 17 05:20:12.894355 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:20:12.894336    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tx9gj_c1baeee2-350c-43c5-9a04-fb34378edb5c/ovn-acl-logging/0.log"
Jul 17 05:20:17.878431 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:20:17.878399    2562 scope.go:117] "RemoveContainer" containerID="fe51587690dd68c6b74e959e0e988c1597fe227d500a3ceb816b0d12d51170ce"
Jul 17 05:20:17.878836 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:20:17.878727    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 05:20:18.695979 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:20:18.695935    2562 generic.go:356] "Generic (PLEG): container finished" podID="370367ab-def2-4733-85db-8983d56c7c96" containerID="0d61389f14db06229f618357143a764cca24674e0c1488ad9923d015468d8fa6" exitCode=1
Jul 17 05:20:18.696149 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:20:18.696005    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck" event={"ID":"370367ab-def2-4733-85db-8983d56c7c96","Type":"ContainerDied","Data":"0d61389f14db06229f618357143a764cca24674e0c1488ad9923d015468d8fa6"}
Jul 17 05:20:18.696149 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:20:18.696065    2562 scope.go:117] "RemoveContainer" containerID="fe51587690dd68c6b74e959e0e988c1597fe227d500a3ceb816b0d12d51170ce"
Jul 17 05:20:18.696409 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:20:18.696392    2562 scope.go:117] "RemoveContainer" containerID="0d61389f14db06229f618357143a764cca24674e0c1488ad9923d015468d8fa6"
Jul 17 05:20:18.696630 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:20:18.696609    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck_odh-ai-gateway-infra(370367ab-def2-4733-85db-8983d56c7c96)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck" podUID="370367ab-def2-4733-85db-8983d56c7c96"
Jul 17 05:20:19.419619 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:20:19.419572    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck"
Jul 17 05:20:19.709075 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:20:19.708989    2562 scope.go:117] "RemoveContainer" containerID="0d61389f14db06229f618357143a764cca24674e0c1488ad9923d015468d8fa6"
Jul 17 05:20:19.709219 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:20:19.709195    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck_odh-ai-gateway-infra(370367ab-def2-4733-85db-8983d56c7c96)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck" podUID="370367ab-def2-4733-85db-8983d56c7c96"
Jul 17 05:20:24.420118 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:20:24.420064    2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck"
Jul 17 05:20:24.420665 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:20:24.420583    2562 scope.go:117] "RemoveContainer" containerID="0d61389f14db06229f618357143a764cca24674e0c1488ad9923d015468d8fa6"
Jul 17 05:20:24.420845 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:20:24.420819    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck_odh-ai-gateway-infra(370367ab-def2-4733-85db-8983d56c7c96)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck" podUID="370367ab-def2-4733-85db-8983d56c7c96"
Jul 17 05:20:38.878632 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:20:38.878593    2562 scope.go:117] "RemoveContainer" containerID="0d61389f14db06229f618357143a764cca24674e0c1488ad9923d015468d8fa6"
Jul 17 05:20:39.812680 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:20:39.812647    2562 generic.go:356] "Generic (PLEG): container finished" podID="370367ab-def2-4733-85db-8983d56c7c96" containerID="ee3f3cb658f8a87d6128dc9e34e669ef6d75983e28eff7a3b61c3b6defc37120" exitCode=1
Jul 17 05:20:39.812861 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:20:39.812722    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck" event={"ID":"370367ab-def2-4733-85db-8983d56c7c96","Type":"ContainerDied","Data":"ee3f3cb658f8a87d6128dc9e34e669ef6d75983e28eff7a3b61c3b6defc37120"}
Jul 17 05:20:39.812861 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:20:39.812780    2562 scope.go:117] "RemoveContainer" containerID="0d61389f14db06229f618357143a764cca24674e0c1488ad9923d015468d8fa6"
Jul 17 05:20:39.813266 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:20:39.813244    2562 scope.go:117] "RemoveContainer" containerID="ee3f3cb658f8a87d6128dc9e34e669ef6d75983e28eff7a3b61c3b6defc37120"
Jul 17 05:20:39.813470 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:20:39.813451    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck_odh-ai-gateway-infra(370367ab-def2-4733-85db-8983d56c7c96)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck" podUID="370367ab-def2-4733-85db-8983d56c7c96"
Jul 17 05:20:44.420105 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:20:44.420066    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck"
Jul 17 05:20:44.420527 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:20:44.420158    2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck"
Jul 17 05:20:44.420527 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:20:44.420497    2562 scope.go:117] "RemoveContainer" containerID="ee3f3cb658f8a87d6128dc9e34e669ef6d75983e28eff7a3b61c3b6defc37120"
Jul 17 05:20:44.420698 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:20:44.420677    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck_odh-ai-gateway-infra(370367ab-def2-4733-85db-8983d56c7c96)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck" podUID="370367ab-def2-4733-85db-8983d56c7c96"
Jul 17 05:20:54.885368 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:20:54.885337    2562 scope.go:117] "RemoveContainer" containerID="ee3f3cb658f8a87d6128dc9e34e669ef6d75983e28eff7a3b61c3b6defc37120"
Jul 17 05:20:54.886215 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:20:54.886176    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck_odh-ai-gateway-infra(370367ab-def2-4733-85db-8983d56c7c96)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck" podUID="370367ab-def2-4733-85db-8983d56c7c96"
Jul 17 05:20:58.435565 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:20:58.435528    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl"]
Jul 17 05:20:58.436073 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:20:58.435733    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl" podUID="e823110b-3d4e-48bf-a436-5a219307d70e" containerName="istio-proxy" containerID="cri-o://c30d19dcede625fd15e6da38cf441d43a205ba8e2fc182fcf32768fb40f32dbe" gracePeriod=30
Jul 17 05:21:03.692661 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:03.692633    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl"
Jul 17 05:21:03.802781 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:03.802743    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e823110b-3d4e-48bf-a436-5a219307d70e-istio-data\") pod \"e823110b-3d4e-48bf-a436-5a219307d70e\" (UID: \"e823110b-3d4e-48bf-a436-5a219307d70e\") "
Jul 17 05:21:03.803005 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:03.802814    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e823110b-3d4e-48bf-a436-5a219307d70e-istio-envoy\") pod \"e823110b-3d4e-48bf-a436-5a219307d70e\" (UID: \"e823110b-3d4e-48bf-a436-5a219307d70e\") "
Jul 17 05:21:03.803005 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:03.802872    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e823110b-3d4e-48bf-a436-5a219307d70e-credential-socket\") pod \"e823110b-3d4e-48bf-a436-5a219307d70e\" (UID: \"e823110b-3d4e-48bf-a436-5a219307d70e\") "
Jul 17 05:21:03.803005 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:03.802898    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e823110b-3d4e-48bf-a436-5a219307d70e-istiod-ca-cert\") pod \"e823110b-3d4e-48bf-a436-5a219307d70e\" (UID: \"e823110b-3d4e-48bf-a436-5a219307d70e\") "
Jul 17 05:21:03.803005 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:03.802918    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e823110b-3d4e-48bf-a436-5a219307d70e-workload-socket\") pod \"e823110b-3d4e-48bf-a436-5a219307d70e\" (UID: \"e823110b-3d4e-48bf-a436-5a219307d70e\") "
Jul 17 05:21:03.803005 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:03.802994    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e823110b-3d4e-48bf-a436-5a219307d70e-istio-token\") pod \"e823110b-3d4e-48bf-a436-5a219307d70e\" (UID: \"e823110b-3d4e-48bf-a436-5a219307d70e\") "
Jul 17 05:21:03.803290 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:03.803029    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dk5qs\" (UniqueName: \"kubernetes.io/projected/e823110b-3d4e-48bf-a436-5a219307d70e-kube-api-access-dk5qs\") pod \"e823110b-3d4e-48bf-a436-5a219307d70e\" (UID: \"e823110b-3d4e-48bf-a436-5a219307d70e\") "
Jul 17 05:21:03.803290 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:03.803091    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e823110b-3d4e-48bf-a436-5a219307d70e-istio-podinfo\") pod \"e823110b-3d4e-48bf-a436-5a219307d70e\" (UID: \"e823110b-3d4e-48bf-a436-5a219307d70e\") "
Jul 17 05:21:03.803290 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:03.803133    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e823110b-3d4e-48bf-a436-5a219307d70e-workload-certs\") pod \"e823110b-3d4e-48bf-a436-5a219307d70e\" (UID: \"e823110b-3d4e-48bf-a436-5a219307d70e\") "
Jul 17 05:21:03.803290 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:03.803221    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e823110b-3d4e-48bf-a436-5a219307d70e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e823110b-3d4e-48bf-a436-5a219307d70e" (UID: "e823110b-3d4e-48bf-a436-5a219307d70e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:21:03.803500 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:03.803299    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e823110b-3d4e-48bf-a436-5a219307d70e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e823110b-3d4e-48bf-a436-5a219307d70e" (UID: "e823110b-3d4e-48bf-a436-5a219307d70e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:21:03.803500 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:03.803367    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e823110b-3d4e-48bf-a436-5a219307d70e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e823110b-3d4e-48bf-a436-5a219307d70e" (UID: "e823110b-3d4e-48bf-a436-5a219307d70e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:21:03.803500 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:03.803449    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e823110b-3d4e-48bf-a436-5a219307d70e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e823110b-3d4e-48bf-a436-5a219307d70e" (UID: "e823110b-3d4e-48bf-a436-5a219307d70e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:21:03.803675 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:03.803611    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e823110b-3d4e-48bf-a436-5a219307d70e-workload-certs\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:21:03.803675 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:03.803630    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e823110b-3d4e-48bf-a436-5a219307d70e-credential-socket\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:21:03.803675 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:03.803644    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e823110b-3d4e-48bf-a436-5a219307d70e-istiod-ca-cert\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:21:03.803675 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:03.803656    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e823110b-3d4e-48bf-a436-5a219307d70e-workload-socket\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:21:03.804060 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:03.803840    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e823110b-3d4e-48bf-a436-5a219307d70e-istio-data" (OuterVolumeSpecName: "istio-data") pod "e823110b-3d4e-48bf-a436-5a219307d70e" (UID: "e823110b-3d4e-48bf-a436-5a219307d70e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:21:03.805470 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:03.805439    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e823110b-3d4e-48bf-a436-5a219307d70e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e823110b-3d4e-48bf-a436-5a219307d70e" (UID: "e823110b-3d4e-48bf-a436-5a219307d70e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 05:21:03.805470 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:03.805456    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e823110b-3d4e-48bf-a436-5a219307d70e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e823110b-3d4e-48bf-a436-5a219307d70e" (UID: "e823110b-3d4e-48bf-a436-5a219307d70e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:21:03.805470 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:03.805452    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e823110b-3d4e-48bf-a436-5a219307d70e-istio-token" (OuterVolumeSpecName: "istio-token") pod "e823110b-3d4e-48bf-a436-5a219307d70e" (UID: "e823110b-3d4e-48bf-a436-5a219307d70e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:21:03.805705 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:03.805452    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e823110b-3d4e-48bf-a436-5a219307d70e-kube-api-access-dk5qs" (OuterVolumeSpecName: "kube-api-access-dk5qs") pod "e823110b-3d4e-48bf-a436-5a219307d70e" (UID: "e823110b-3d4e-48bf-a436-5a219307d70e"). InnerVolumeSpecName "kube-api-access-dk5qs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:21:03.904594 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:03.904546    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e823110b-3d4e-48bf-a436-5a219307d70e-istio-envoy\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:21:03.904594 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:03.904583    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e823110b-3d4e-48bf-a436-5a219307d70e-istio-token\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:21:03.904594 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:03.904593    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dk5qs\" (UniqueName: \"kubernetes.io/projected/e823110b-3d4e-48bf-a436-5a219307d70e-kube-api-access-dk5qs\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:21:03.904594 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:03.904602    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e823110b-3d4e-48bf-a436-5a219307d70e-istio-podinfo\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:21:03.904882 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:03.904612    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e823110b-3d4e-48bf-a436-5a219307d70e-istio-data\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:21:03.936880 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:03.936847    2562 generic.go:356] "Generic (PLEG): container finished" podID="e823110b-3d4e-48bf-a436-5a219307d70e" containerID="c30d19dcede625fd15e6da38cf441d43a205ba8e2fc182fcf32768fb40f32dbe" exitCode=0
Jul 17 05:21:03.937030 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:03.936900    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl"
Jul 17 05:21:03.937030 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:03.936937    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl" event={"ID":"e823110b-3d4e-48bf-a436-5a219307d70e","Type":"ContainerDied","Data":"c30d19dcede625fd15e6da38cf441d43a205ba8e2fc182fcf32768fb40f32dbe"}
Jul 17 05:21:03.937030 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:03.937021    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl" event={"ID":"e823110b-3d4e-48bf-a436-5a219307d70e","Type":"ContainerDied","Data":"222e3052b7df2bc1c74b241bb3b26ee029e3b5dc873e3aa729ad850a9341a035"}
Jul 17 05:21:03.937155 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:03.937052    2562 scope.go:117] "RemoveContainer" containerID="c30d19dcede625fd15e6da38cf441d43a205ba8e2fc182fcf32768fb40f32dbe"
Jul 17 05:21:03.946121 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:03.946104    2562 scope.go:117] "RemoveContainer" containerID="c4ce9b45312b8a605d5a6163bb8469b43cc60859f5bc4a366fdc482995545221"
Jul 17 05:21:03.954342 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:03.954323    2562 scope.go:117] "RemoveContainer" containerID="c30d19dcede625fd15e6da38cf441d43a205ba8e2fc182fcf32768fb40f32dbe"
Jul 17 05:21:03.954592 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:21:03.954573    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c30d19dcede625fd15e6da38cf441d43a205ba8e2fc182fcf32768fb40f32dbe\": container with ID starting with c30d19dcede625fd15e6da38cf441d43a205ba8e2fc182fcf32768fb40f32dbe not found: ID does not exist" containerID="c30d19dcede625fd15e6da38cf441d43a205ba8e2fc182fcf32768fb40f32dbe"
Jul 17 05:21:03.954645 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:03.954601    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c30d19dcede625fd15e6da38cf441d43a205ba8e2fc182fcf32768fb40f32dbe"} err="failed to get container status \"c30d19dcede625fd15e6da38cf441d43a205ba8e2fc182fcf32768fb40f32dbe\": rpc error: code = NotFound desc = could not find container \"c30d19dcede625fd15e6da38cf441d43a205ba8e2fc182fcf32768fb40f32dbe\": container with ID starting with c30d19dcede625fd15e6da38cf441d43a205ba8e2fc182fcf32768fb40f32dbe not found: ID does not exist"
Jul 17 05:21:03.954645 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:03.954619    2562 scope.go:117] "RemoveContainer" containerID="c4ce9b45312b8a605d5a6163bb8469b43cc60859f5bc4a366fdc482995545221"
Jul 17 05:21:03.954852 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:21:03.954837    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4ce9b45312b8a605d5a6163bb8469b43cc60859f5bc4a366fdc482995545221\": container with ID starting with c4ce9b45312b8a605d5a6163bb8469b43cc60859f5bc4a366fdc482995545221 not found: ID does not exist" containerID="c4ce9b45312b8a605d5a6163bb8469b43cc60859f5bc4a366fdc482995545221"
Jul 17 05:21:03.954899 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:03.954854    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4ce9b45312b8a605d5a6163bb8469b43cc60859f5bc4a366fdc482995545221"} err="failed to get container status \"c4ce9b45312b8a605d5a6163bb8469b43cc60859f5bc4a366fdc482995545221\": rpc error: code = NotFound desc = could not find container \"c4ce9b45312b8a605d5a6163bb8469b43cc60859f5bc4a366fdc482995545221\": container with ID starting with c4ce9b45312b8a605d5a6163bb8469b43cc60859f5bc4a366fdc482995545221 not found: ID does not exist"
Jul 17 05:21:03.958689 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:03.958665    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl"]
Jul 17 05:21:03.962405 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:03.962382    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-c8a383ce-gw-openshift-default-7cb9d8587f-ddkvl"]
Jul 17 05:21:04.481257 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:04.481216    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck"]
Jul 17 05:21:04.631177 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:04.631153    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck"
Jul 17 05:21:04.714120 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:04.714082    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/370367ab-def2-4733-85db-8983d56c7c96-maas-api-tls\") pod \"370367ab-def2-4733-85db-8983d56c7c96\" (UID: \"370367ab-def2-4733-85db-8983d56c7c96\") "
Jul 17 05:21:04.714610 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:04.714154    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zcvfw\" (UniqueName: \"kubernetes.io/projected/370367ab-def2-4733-85db-8983d56c7c96-kube-api-access-zcvfw\") pod \"370367ab-def2-4733-85db-8983d56c7c96\" (UID: \"370367ab-def2-4733-85db-8983d56c7c96\") "
Jul 17 05:21:04.714610 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:04.714239    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/370367ab-def2-4733-85db-8983d56c7c96-openshift-service-ca\") pod \"370367ab-def2-4733-85db-8983d56c7c96\" (UID: \"370367ab-def2-4733-85db-8983d56c7c96\") "
Jul 17 05:21:04.714728 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:04.714652    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/370367ab-def2-4733-85db-8983d56c7c96-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "370367ab-def2-4733-85db-8983d56c7c96" (UID: "370367ab-def2-4733-85db-8983d56c7c96"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:21:04.716501 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:04.716476    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/370367ab-def2-4733-85db-8983d56c7c96-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "370367ab-def2-4733-85db-8983d56c7c96" (UID: "370367ab-def2-4733-85db-8983d56c7c96"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 05:21:04.716501 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:04.716488    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/370367ab-def2-4733-85db-8983d56c7c96-kube-api-access-zcvfw" (OuterVolumeSpecName: "kube-api-access-zcvfw") pod "370367ab-def2-4733-85db-8983d56c7c96" (UID: "370367ab-def2-4733-85db-8983d56c7c96"). InnerVolumeSpecName "kube-api-access-zcvfw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:21:04.815617 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:04.815517    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/370367ab-def2-4733-85db-8983d56c7c96-maas-api-tls\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:21:04.815617 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:04.815553    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zcvfw\" (UniqueName: \"kubernetes.io/projected/370367ab-def2-4733-85db-8983d56c7c96-kube-api-access-zcvfw\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:21:04.815617 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:04.815564    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/370367ab-def2-4733-85db-8983d56c7c96-openshift-service-ca\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:21:04.884441 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:04.884401    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e823110b-3d4e-48bf-a436-5a219307d70e" path="/var/lib/kubelet/pods/e823110b-3d4e-48bf-a436-5a219307d70e/volumes"
Jul 17 05:21:04.943627 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:04.943598    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck"
Jul 17 05:21:04.943812 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:04.943602    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck" event={"ID":"370367ab-def2-4733-85db-8983d56c7c96","Type":"ContainerDied","Data":"bf2671f53cc2101fa01aaa033e5a635bd93c5c7083a68eda3435906f18ff516d"}
Jul 17 05:21:04.943812 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:04.943748    2562 scope.go:117] "RemoveContainer" containerID="ee3f3cb658f8a87d6128dc9e34e669ef6d75983e28eff7a3b61c3b6defc37120"
Jul 17 05:21:04.960185 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:04.960156    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck"]
Jul 17 05:21:04.963975 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:04.963939    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c8a383ce-7ddcc5dbcc-gzfck"]
Jul 17 05:21:06.884089 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:06.884054    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="370367ab-def2-4733-85db-8983d56c7c96" path="/var/lib/kubelet/pods/370367ab-def2-4733-85db-8983d56c7c96/volumes"
Jul 17 05:21:10.254394 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.254361    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc"]
Jul 17 05:21:10.255166 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.255136    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e823110b-3d4e-48bf-a436-5a219307d70e" containerName="istio-proxy"
Jul 17 05:21:10.255166 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.255162    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e823110b-3d4e-48bf-a436-5a219307d70e" containerName="istio-proxy"
Jul 17 05:21:10.255318 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.255175    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="370367ab-def2-4733-85db-8983d56c7c96" containerName="maas-api"
Jul 17 05:21:10.255318 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.255183    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="370367ab-def2-4733-85db-8983d56c7c96" containerName="maas-api"
Jul 17 05:21:10.255318 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.255198    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="370367ab-def2-4733-85db-8983d56c7c96" containerName="maas-api"
Jul 17 05:21:10.255318 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.255208    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="370367ab-def2-4733-85db-8983d56c7c96" containerName="maas-api"
Jul 17 05:21:10.255318 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.255221    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e823110b-3d4e-48bf-a436-5a219307d70e" containerName="istio-proxy"
Jul 17 05:21:10.255318 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.255229    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e823110b-3d4e-48bf-a436-5a219307d70e" containerName="istio-proxy"
Jul 17 05:21:10.255318 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.255250    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e7718a9-2705-4130-864b-6d1a9df9c5d6" containerName="maas-api"
Jul 17 05:21:10.255318 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.255258    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e7718a9-2705-4130-864b-6d1a9df9c5d6" containerName="maas-api"
Jul 17 05:21:10.255706 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.255373    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="370367ab-def2-4733-85db-8983d56c7c96" containerName="maas-api"
Jul 17 05:21:10.255706 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.255386    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="370367ab-def2-4733-85db-8983d56c7c96" containerName="maas-api"
Jul 17 05:21:10.255706 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.255396    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e823110b-3d4e-48bf-a436-5a219307d70e" containerName="istio-proxy"
Jul 17 05:21:10.255706 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.255408    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="370367ab-def2-4733-85db-8983d56c7c96" containerName="maas-api"
Jul 17 05:21:10.258561 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.258536    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc"
Jul 17 05:21:10.262023 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.261998    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-9eb8486e-openshift-default-dockercfg-7qvr8\""
Jul 17 05:21:10.273407 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.273370    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc"]
Jul 17 05:21:10.368828 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.368791    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cdb1b691-5f72-44e8-ae99-afcddb755ce3-istio-token\") pod \"e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc\" (UID: \"cdb1b691-5f72-44e8-ae99-afcddb755ce3\") " pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc"
Jul 17 05:21:10.369061 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.368839    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cdb1b691-5f72-44e8-ae99-afcddb755ce3-istio-envoy\") pod \"e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc\" (UID: \"cdb1b691-5f72-44e8-ae99-afcddb755ce3\") " pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc"
Jul 17 05:21:10.369061 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.368874    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdb1b691-5f72-44e8-ae99-afcddb755ce3-workload-socket\") pod \"e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc\" (UID: \"cdb1b691-5f72-44e8-ae99-afcddb755ce3\") " pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc"
Jul 17 05:21:10.369061 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.368905    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cdb1b691-5f72-44e8-ae99-afcddb755ce3-workload-certs\") pod \"e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc\" (UID: \"cdb1b691-5f72-44e8-ae99-afcddb755ce3\") " pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc"
Jul 17 05:21:10.369262 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.369053    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdb1b691-5f72-44e8-ae99-afcddb755ce3-credential-socket\") pod \"e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc\" (UID: \"cdb1b691-5f72-44e8-ae99-afcddb755ce3\") " pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc"
Jul 17 05:21:10.369262 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.369096    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cdb1b691-5f72-44e8-ae99-afcddb755ce3-istio-data\") pod \"e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc\" (UID: \"cdb1b691-5f72-44e8-ae99-afcddb755ce3\") " pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc"
Jul 17 05:21:10.369262 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.369147    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cdb1b691-5f72-44e8-ae99-afcddb755ce3-istiod-ca-cert\") pod \"e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc\" (UID: \"cdb1b691-5f72-44e8-ae99-afcddb755ce3\") " pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc"
Jul 17 05:21:10.369262 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.369188    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cdb1b691-5f72-44e8-ae99-afcddb755ce3-istio-podinfo\") pod \"e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc\" (UID: \"cdb1b691-5f72-44e8-ae99-afcddb755ce3\") " pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc"
Jul 17 05:21:10.369262 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.369225    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s979f\" (UniqueName: \"kubernetes.io/projected/cdb1b691-5f72-44e8-ae99-afcddb755ce3-kube-api-access-s979f\") pod \"e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc\" (UID: \"cdb1b691-5f72-44e8-ae99-afcddb755ce3\") " pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc"
Jul 17 05:21:10.470089 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.470046    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cdb1b691-5f72-44e8-ae99-afcddb755ce3-istiod-ca-cert\") pod \"e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc\" (UID: \"cdb1b691-5f72-44e8-ae99-afcddb755ce3\") " pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc"
Jul 17 05:21:10.470089 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.470095    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cdb1b691-5f72-44e8-ae99-afcddb755ce3-istio-podinfo\") pod \"e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc\" (UID: \"cdb1b691-5f72-44e8-ae99-afcddb755ce3\") " pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc"
Jul 17 05:21:10.470314 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.470225    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s979f\" (UniqueName: \"kubernetes.io/projected/cdb1b691-5f72-44e8-ae99-afcddb755ce3-kube-api-access-s979f\") pod \"e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc\" (UID: \"cdb1b691-5f72-44e8-ae99-afcddb755ce3\") " pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc"
Jul 17 05:21:10.470359 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.470339    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cdb1b691-5f72-44e8-ae99-afcddb755ce3-istio-token\") pod \"e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc\" (UID: \"cdb1b691-5f72-44e8-ae99-afcddb755ce3\") " pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc"
Jul 17 05:21:10.470394 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.470384    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cdb1b691-5f72-44e8-ae99-afcddb755ce3-istio-envoy\") pod \"e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc\" (UID: \"cdb1b691-5f72-44e8-ae99-afcddb755ce3\") " pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc"
Jul 17 05:21:10.470537 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.470510    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdb1b691-5f72-44e8-ae99-afcddb755ce3-workload-socket\") pod \"e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc\" (UID: \"cdb1b691-5f72-44e8-ae99-afcddb755ce3\") " pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc"
Jul 17 05:21:10.470601 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.470561    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cdb1b691-5f72-44e8-ae99-afcddb755ce3-workload-certs\") pod \"e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc\" (UID: \"cdb1b691-5f72-44e8-ae99-afcddb755ce3\") " pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc"
Jul 17 05:21:10.470661 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.470633    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdb1b691-5f72-44e8-ae99-afcddb755ce3-credential-socket\") pod \"e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc\" (UID: \"cdb1b691-5f72-44e8-ae99-afcddb755ce3\") " pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc"
Jul 17 05:21:10.470726 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.470661    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cdb1b691-5f72-44e8-ae99-afcddb755ce3-istio-data\") pod \"e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc\" (UID: \"cdb1b691-5f72-44e8-ae99-afcddb755ce3\") " pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc"
Jul 17 05:21:10.470846 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.470820    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdb1b691-5f72-44e8-ae99-afcddb755ce3-workload-socket\") pod \"e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc\" (UID: \"cdb1b691-5f72-44e8-ae99-afcddb755ce3\") " pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc"
Jul 17 05:21:10.470958 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.470909    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cdb1b691-5f72-44e8-ae99-afcddb755ce3-istiod-ca-cert\") pod \"e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc\" (UID: \"cdb1b691-5f72-44e8-ae99-afcddb755ce3\") " pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc"
Jul 17 05:21:10.471273 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.471019    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cdb1b691-5f72-44e8-ae99-afcddb755ce3-workload-certs\") pod \"e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc\" (UID: \"cdb1b691-5f72-44e8-ae99-afcddb755ce3\") " pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc"
Jul 17 05:21:10.471273 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.471029    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cdb1b691-5f72-44e8-ae99-afcddb755ce3-istio-data\") pod \"e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc\" (UID: \"cdb1b691-5f72-44e8-ae99-afcddb755ce3\") " pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc"
Jul 17 05:21:10.471273 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.471204    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdb1b691-5f72-44e8-ae99-afcddb755ce3-credential-socket\") pod \"e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc\" (UID: \"cdb1b691-5f72-44e8-ae99-afcddb755ce3\") " pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc"
Jul 17 05:21:10.472643 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.472622    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cdb1b691-5f72-44e8-ae99-afcddb755ce3-istio-envoy\") pod \"e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc\" (UID: \"cdb1b691-5f72-44e8-ae99-afcddb755ce3\") " pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc"
Jul 17 05:21:10.472757 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.472740    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cdb1b691-5f72-44e8-ae99-afcddb755ce3-istio-podinfo\") pod \"e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc\" (UID: \"cdb1b691-5f72-44e8-ae99-afcddb755ce3\") " pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc"
Jul 17 05:21:10.483723 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.483695    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s979f\" (UniqueName: \"kubernetes.io/projected/cdb1b691-5f72-44e8-ae99-afcddb755ce3-kube-api-access-s979f\") pod \"e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc\" (UID: \"cdb1b691-5f72-44e8-ae99-afcddb755ce3\") " pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc"
Jul 17 05:21:10.486316 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.486298    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cdb1b691-5f72-44e8-ae99-afcddb755ce3-istio-token\") pod \"e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc\" (UID: \"cdb1b691-5f72-44e8-ae99-afcddb755ce3\") " pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc"
Jul 17 05:21:10.572320 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.572288    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc"
Jul 17 05:21:10.713164 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.713137    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc"]
Jul 17 05:21:10.715148 ip-10-0-132-19 kubenswrapper[2562]: W0717 05:21:10.715111    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdb1b691_5f72_44e8_ae99_afcddb755ce3.slice/crio-a63160e76f60a0496e63db7ccb8c66a9bb0b5af5b5532a64e78c67ec94c64733 WatchSource:0}: Error finding container a63160e76f60a0496e63db7ccb8c66a9bb0b5af5b5532a64e78c67ec94c64733: Status 404 returned error can't find the container with id a63160e76f60a0496e63db7ccb8c66a9bb0b5af5b5532a64e78c67ec94c64733
Jul 17 05:21:10.717158 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.717126    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:21:10.717230 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.717199    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:21:10.717230 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.717226    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:21:10.977577 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.977478    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc" event={"ID":"cdb1b691-5f72-44e8-ae99-afcddb755ce3","Type":"ContainerStarted","Data":"9f9758767f97f5baeb389a8103d5e3dc6c2bc7a146c7830259ca3b3f34a037d6"}
Jul 17 05:21:10.977577 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.977538    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc" event={"ID":"cdb1b691-5f72-44e8-ae99-afcddb755ce3","Type":"ContainerStarted","Data":"a63160e76f60a0496e63db7ccb8c66a9bb0b5af5b5532a64e78c67ec94c64733"}
Jul 17 05:21:10.999452 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:10.999394    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc" podStartSLOduration=0.999375996 podStartE2EDuration="999.375996ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:21:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:21:10.996912069 +0000 UTC m=+1858.680505749" watchObservedRunningTime="2026-07-17 05:21:10.999375996 +0000 UTC m=+1858.682969653"
Jul 17 05:21:11.510824 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:11.510766    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm"]
Jul 17 05:21:11.511476 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:11.511454    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="370367ab-def2-4733-85db-8983d56c7c96" containerName="maas-api"
Jul 17 05:21:11.511536 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:11.511481    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="370367ab-def2-4733-85db-8983d56c7c96" containerName="maas-api"
Jul 17 05:21:11.511536 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:11.511499    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="370367ab-def2-4733-85db-8983d56c7c96" containerName="maas-api"
Jul 17 05:21:11.511536 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:11.511511    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="370367ab-def2-4733-85db-8983d56c7c96" containerName="maas-api"
Jul 17 05:21:11.511630 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:11.511609    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="370367ab-def2-4733-85db-8983d56c7c96" containerName="maas-api"
Jul 17 05:21:11.511630 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:11.511625    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e823110b-3d4e-48bf-a436-5a219307d70e" containerName="istio-proxy"
Jul 17 05:21:11.514327 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:11.514306    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm"
Jul 17 05:21:11.516841 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:11.516819    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-9eb8486e-serving-cert\""
Jul 17 05:21:11.516955 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:11.516897    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-xwcd7\""
Jul 17 05:21:11.525604 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:11.525578    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm"]
Jul 17 05:21:11.573055 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:11.573017    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc"
Jul 17 05:21:11.578222 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:11.578198    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc"
Jul 17 05:21:11.582205 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:11.582180    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbzl7\" (UniqueName: \"kubernetes.io/projected/34c6ad80-2bd4-4768-94ee-c91a703f9f4d-kube-api-access-kbzl7\") pod \"maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm\" (UID: \"34c6ad80-2bd4-4768-94ee-c91a703f9f4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm"
Jul 17 05:21:11.582328 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:11.582222    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/34c6ad80-2bd4-4768-94ee-c91a703f9f4d-maas-api-tls\") pod \"maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm\" (UID: \"34c6ad80-2bd4-4768-94ee-c91a703f9f4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm"
Jul 17 05:21:11.582328 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:11.582316    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/34c6ad80-2bd4-4768-94ee-c91a703f9f4d-openshift-service-ca\") pod \"maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm\" (UID: \"34c6ad80-2bd4-4768-94ee-c91a703f9f4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm"
Jul 17 05:21:11.683070 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:11.683032    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/34c6ad80-2bd4-4768-94ee-c91a703f9f4d-openshift-service-ca\") pod \"maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm\" (UID: \"34c6ad80-2bd4-4768-94ee-c91a703f9f4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm"
Jul 17 05:21:11.683260 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:11.683097    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbzl7\" (UniqueName: \"kubernetes.io/projected/34c6ad80-2bd4-4768-94ee-c91a703f9f4d-kube-api-access-kbzl7\") pod \"maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm\" (UID: \"34c6ad80-2bd4-4768-94ee-c91a703f9f4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm"
Jul 17 05:21:11.683260 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:11.683124    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/34c6ad80-2bd4-4768-94ee-c91a703f9f4d-maas-api-tls\") pod \"maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm\" (UID: \"34c6ad80-2bd4-4768-94ee-c91a703f9f4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm"
Jul 17 05:21:11.683815 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:11.683797    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/34c6ad80-2bd4-4768-94ee-c91a703f9f4d-openshift-service-ca\") pod \"maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm\" (UID: \"34c6ad80-2bd4-4768-94ee-c91a703f9f4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm"
Jul 17 05:21:11.685803 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:11.685781    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/34c6ad80-2bd4-4768-94ee-c91a703f9f4d-maas-api-tls\") pod \"maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm\" (UID: \"34c6ad80-2bd4-4768-94ee-c91a703f9f4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm"
Jul 17 05:21:11.693104 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:11.693081    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbzl7\" (UniqueName: \"kubernetes.io/projected/34c6ad80-2bd4-4768-94ee-c91a703f9f4d-kube-api-access-kbzl7\") pod \"maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm\" (UID: \"34c6ad80-2bd4-4768-94ee-c91a703f9f4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm"
Jul 17 05:21:11.827051 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:11.827014    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm"
Jul 17 05:21:11.962092 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:11.962070    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm"]
Jul 17 05:21:11.963888 ip-10-0-132-19 kubenswrapper[2562]: W0717 05:21:11.963862    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34c6ad80_2bd4_4768_94ee_c91a703f9f4d.slice/crio-a12611038dc6153072dcc53087715f3cf17fde221159f5152f53e72800a677d8 WatchSource:0}: Error finding container a12611038dc6153072dcc53087715f3cf17fde221159f5152f53e72800a677d8: Status 404 returned error can't find the container with id a12611038dc6153072dcc53087715f3cf17fde221159f5152f53e72800a677d8
Jul 17 05:21:11.983499 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:11.983395    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" event={"ID":"34c6ad80-2bd4-4768-94ee-c91a703f9f4d","Type":"ContainerStarted","Data":"a12611038dc6153072dcc53087715f3cf17fde221159f5152f53e72800a677d8"}
Jul 17 05:21:11.983692 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:11.983678    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc"
Jul 17 05:21:11.984841 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:11.984820    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc"
Jul 17 05:21:12.989801 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:12.989761    2562 generic.go:356] "Generic (PLEG): container finished" podID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d" containerID="876043685cebf3c95861bd93722e50299d9ecd96cbde2b4eabeb80e51230b011" exitCode=1
Jul 17 05:21:12.990170 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:12.989800    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" event={"ID":"34c6ad80-2bd4-4768-94ee-c91a703f9f4d","Type":"ContainerDied","Data":"876043685cebf3c95861bd93722e50299d9ecd96cbde2b4eabeb80e51230b011"}
Jul 17 05:21:12.990228 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:12.990207    2562 scope.go:117] "RemoveContainer" containerID="876043685cebf3c95861bd93722e50299d9ecd96cbde2b4eabeb80e51230b011"
Jul 17 05:21:13.997337 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:13.997301    2562 generic.go:356] "Generic (PLEG): container finished" podID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d" containerID="0bc83a4ebe428a5ffdb65df927e756a7547bea4dd3cd0eed1fbdd8da8ae5f64a" exitCode=1
Jul 17 05:21:13.997752 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:13.997388    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" event={"ID":"34c6ad80-2bd4-4768-94ee-c91a703f9f4d","Type":"ContainerDied","Data":"0bc83a4ebe428a5ffdb65df927e756a7547bea4dd3cd0eed1fbdd8da8ae5f64a"}
Jul 17 05:21:13.997752 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:13.997461    2562 scope.go:117] "RemoveContainer" containerID="876043685cebf3c95861bd93722e50299d9ecd96cbde2b4eabeb80e51230b011"
Jul 17 05:21:13.997752 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:13.997699    2562 scope.go:117] "RemoveContainer" containerID="0bc83a4ebe428a5ffdb65df927e756a7547bea4dd3cd0eed1fbdd8da8ae5f64a"
Jul 17 05:21:13.998014 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:21:13.997947    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm_odh-ai-gateway-infra(34c6ad80-2bd4-4768-94ee-c91a703f9f4d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d"
Jul 17 05:21:15.003713 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:15.003680    2562 scope.go:117] "RemoveContainer" containerID="0bc83a4ebe428a5ffdb65df927e756a7547bea4dd3cd0eed1fbdd8da8ae5f64a"
Jul 17 05:21:15.004200 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:21:15.003896    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm_odh-ai-gateway-infra(34c6ad80-2bd4-4768-94ee-c91a703f9f4d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d"
Jul 17 05:21:16.828046 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:16.828012    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm"
Jul 17 05:21:16.828445 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:16.828375    2562 scope.go:117] "RemoveContainer" containerID="0bc83a4ebe428a5ffdb65df927e756a7547bea4dd3cd0eed1fbdd8da8ae5f64a"
Jul 17 05:21:16.828596 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:21:16.828576    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm_odh-ai-gateway-infra(34c6ad80-2bd4-4768-94ee-c91a703f9f4d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d"
Jul 17 05:21:21.827936 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:21.827896    2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm"
Jul 17 05:21:21.828385 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:21.828358    2562 scope.go:117] "RemoveContainer" containerID="0bc83a4ebe428a5ffdb65df927e756a7547bea4dd3cd0eed1fbdd8da8ae5f64a"
Jul 17 05:21:21.828552 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:21:21.828530    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm_odh-ai-gateway-infra(34c6ad80-2bd4-4768-94ee-c91a703f9f4d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d"
Jul 17 05:21:23.044897 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:23.044864    2562 generic.go:356] "Generic (PLEG): container finished" podID="cdb1b691-5f72-44e8-ae99-afcddb755ce3" containerID="9f9758767f97f5baeb389a8103d5e3dc6c2bc7a146c7830259ca3b3f34a037d6" exitCode=0
Jul 17 05:21:23.045345 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:23.044943    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc" event={"ID":"cdb1b691-5f72-44e8-ae99-afcddb755ce3","Type":"ContainerDied","Data":"9f9758767f97f5baeb389a8103d5e3dc6c2bc7a146c7830259ca3b3f34a037d6"}
Jul 17 05:21:23.045416 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:23.045401    2562 scope.go:117] "RemoveContainer" containerID="9f9758767f97f5baeb389a8103d5e3dc6c2bc7a146c7830259ca3b3f34a037d6"
Jul 17 05:21:23.046497 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:23.046472    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:21:23.046553 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:23.046542    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:21:23.046597 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:23.046575    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:21:24.051887 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:24.051843    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc" event={"ID":"cdb1b691-5f72-44e8-ae99-afcddb755ce3","Type":"ContainerStarted","Data":"bd9fb856d43a741919bea4356dd19ed1c88461ebf2d2c4e2f9b8e107ece8249b"}
Jul 17 05:21:24.574837 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:24.574803    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc"
Jul 17 05:21:24.575292 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:24.575263    2562 patch_prober.go:28] interesting pod/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Jul 17 05:21:24.575504 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:24.575480    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc" podUID="cdb1b691-5f72-44e8-ae99-afcddb755ce3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Jul 17 05:21:25.573093 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:25.573049    2562 patch_prober.go:28] interesting pod/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Jul 17 05:21:25.573734 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:25.573696    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc" podUID="cdb1b691-5f72-44e8-ae99-afcddb755ce3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Jul 17 05:21:26.573052 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:26.573012    2562 patch_prober.go:28] interesting pod/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Jul 17 05:21:26.573356 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:26.573326    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc" podUID="cdb1b691-5f72-44e8-ae99-afcddb755ce3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Jul 17 05:21:26.984214 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:26.984121    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc"
Jul 17 05:21:27.573760 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:27.573716    2562 patch_prober.go:28] interesting pod/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Jul 17 05:21:27.574443 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:27.574403    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc" podUID="cdb1b691-5f72-44e8-ae99-afcddb755ce3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Jul 17 05:21:28.576877 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:28.576845    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc"
Jul 17 05:21:28.577861 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:28.577843    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc"
Jul 17 05:21:33.878501 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:33.878421    2562 scope.go:117] "RemoveContainer" containerID="0bc83a4ebe428a5ffdb65df927e756a7547bea4dd3cd0eed1fbdd8da8ae5f64a"
Jul 17 05:21:35.120380 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:35.120330    2562 generic.go:356] "Generic (PLEG): container finished" podID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d" containerID="9e1ca6b5ce2057713421e5ad486e9c1e97044871fcc7f7ce437f82c73eaf32bf" exitCode=1
Jul 17 05:21:35.120813 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:35.120396    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" event={"ID":"34c6ad80-2bd4-4768-94ee-c91a703f9f4d","Type":"ContainerDied","Data":"9e1ca6b5ce2057713421e5ad486e9c1e97044871fcc7f7ce437f82c73eaf32bf"}
Jul 17 05:21:35.120813 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:35.120466    2562 scope.go:117] "RemoveContainer" containerID="0bc83a4ebe428a5ffdb65df927e756a7547bea4dd3cd0eed1fbdd8da8ae5f64a"
Jul 17 05:21:35.120915 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:35.120860    2562 scope.go:117] "RemoveContainer" containerID="9e1ca6b5ce2057713421e5ad486e9c1e97044871fcc7f7ce437f82c73eaf32bf"
Jul 17 05:21:35.121157 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:21:35.121138    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm_odh-ai-gateway-infra(34c6ad80-2bd4-4768-94ee-c91a703f9f4d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d"
Jul 17 05:21:36.827108 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:36.827067    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm"
Jul 17 05:21:36.827516 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:36.827489    2562 scope.go:117] "RemoveContainer" containerID="9e1ca6b5ce2057713421e5ad486e9c1e97044871fcc7f7ce437f82c73eaf32bf"
Jul 17 05:21:36.827686 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:21:36.827666    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm_odh-ai-gateway-infra(34c6ad80-2bd4-4768-94ee-c91a703f9f4d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d"
Jul 17 05:21:41.827823 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:41.827765    2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm"
Jul 17 05:21:41.828333 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:41.828239    2562 scope.go:117] "RemoveContainer" containerID="9e1ca6b5ce2057713421e5ad486e9c1e97044871fcc7f7ce437f82c73eaf32bf"
Jul 17 05:21:41.828452 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:21:41.828430    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm_odh-ai-gateway-infra(34c6ad80-2bd4-4768-94ee-c91a703f9f4d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d"
Jul 17 05:21:53.878584 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:21:53.878553    2562 scope.go:117] "RemoveContainer" containerID="9e1ca6b5ce2057713421e5ad486e9c1e97044871fcc7f7ce437f82c73eaf32bf"
Jul 17 05:21:53.879066 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:21:53.878827    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm_odh-ai-gateway-infra(34c6ad80-2bd4-4768-94ee-c91a703f9f4d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d"
Jul 17 05:22:07.878592 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:22:07.878563    2562 scope.go:117] "RemoveContainer" containerID="9e1ca6b5ce2057713421e5ad486e9c1e97044871fcc7f7ce437f82c73eaf32bf"
Jul 17 05:22:09.289574 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:22:09.289543    2562 generic.go:356] "Generic (PLEG): container finished" podID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d" containerID="80092349703cf91098dad4434602695a9206c3e9fd487d0c1c4d1ecfe4ac3020" exitCode=1
Jul 17 05:22:09.290052 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:22:09.289621    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" event={"ID":"34c6ad80-2bd4-4768-94ee-c91a703f9f4d","Type":"ContainerDied","Data":"80092349703cf91098dad4434602695a9206c3e9fd487d0c1c4d1ecfe4ac3020"}
Jul 17 05:22:09.290052 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:22:09.289698    2562 scope.go:117] "RemoveContainer" containerID="9e1ca6b5ce2057713421e5ad486e9c1e97044871fcc7f7ce437f82c73eaf32bf"
Jul 17 05:22:09.290162 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:22:09.290080    2562 scope.go:117] "RemoveContainer" containerID="80092349703cf91098dad4434602695a9206c3e9fd487d0c1c4d1ecfe4ac3020"
Jul 17 05:22:09.290330 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:22:09.290312    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm_odh-ai-gateway-infra(34c6ad80-2bd4-4768-94ee-c91a703f9f4d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d"
Jul 17 05:22:11.827547 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:22:11.827513    2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm"
Jul 17 05:22:11.827940 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:22:11.827596    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm"
Jul 17 05:22:11.828010 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:22:11.827943    2562 scope.go:117] "RemoveContainer" containerID="80092349703cf91098dad4434602695a9206c3e9fd487d0c1c4d1ecfe4ac3020"
Jul 17 05:22:11.828216 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:22:11.828194    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm_odh-ai-gateway-infra(34c6ad80-2bd4-4768-94ee-c91a703f9f4d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d"
Jul 17 05:22:24.878808 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:22:24.878775    2562 scope.go:117] "RemoveContainer" containerID="80092349703cf91098dad4434602695a9206c3e9fd487d0c1c4d1ecfe4ac3020"
Jul 17 05:22:24.879302 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:22:24.879055    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm_odh-ai-gateway-infra(34c6ad80-2bd4-4768-94ee-c91a703f9f4d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d"
Jul 17 05:22:35.878126 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:22:35.878089    2562 scope.go:117] "RemoveContainer" containerID="80092349703cf91098dad4434602695a9206c3e9fd487d0c1c4d1ecfe4ac3020"
Jul 17 05:22:35.878639 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:22:35.878294    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm_odh-ai-gateway-infra(34c6ad80-2bd4-4768-94ee-c91a703f9f4d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d"
Jul 17 05:22:48.878759 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:22:48.878719    2562 scope.go:117] "RemoveContainer" containerID="80092349703cf91098dad4434602695a9206c3e9fd487d0c1c4d1ecfe4ac3020"
Jul 17 05:22:49.505597 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:22:49.505510    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" event={"ID":"34c6ad80-2bd4-4768-94ee-c91a703f9f4d","Type":"ContainerStarted","Data":"d4c4950c3164638d8d77bc03b5d90d9f2d54fcd5cb93584b12db36a271a07b95"}
Jul 17 05:22:49.505760 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:22:49.505718    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm"
Jul 17 05:22:49.523587 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:22:49.523520    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" podStartSLOduration=96.49221504 podStartE2EDuration="1m38.523489486s" totalImagesPullingTime="2.031274446s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:21:11 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:22:49.523008545 +0000 UTC m=+1957.206602193" watchObservedRunningTime="2026-07-17 05:22:49.523489486 +0000 UTC m=+1957.207083145"
Jul 17 05:22:50.513274 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:22:50.513238    2562 generic.go:356] "Generic (PLEG): container finished" podID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d" containerID="d4c4950c3164638d8d77bc03b5d90d9f2d54fcd5cb93584b12db36a271a07b95" exitCode=1
Jul 17 05:22:50.513698 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:22:50.513318    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" event={"ID":"34c6ad80-2bd4-4768-94ee-c91a703f9f4d","Type":"ContainerDied","Data":"d4c4950c3164638d8d77bc03b5d90d9f2d54fcd5cb93584b12db36a271a07b95"}
Jul 17 05:22:50.513698 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:22:50.513383    2562 scope.go:117] "RemoveContainer" containerID="80092349703cf91098dad4434602695a9206c3e9fd487d0c1c4d1ecfe4ac3020"
Jul 17 05:22:50.513698 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:22:50.513649    2562 scope.go:117] "RemoveContainer" containerID="d4c4950c3164638d8d77bc03b5d90d9f2d54fcd5cb93584b12db36a271a07b95"
Jul 17 05:22:50.513888 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:22:50.513864    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm_odh-ai-gateway-infra(34c6ad80-2bd4-4768-94ee-c91a703f9f4d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d"
Jul 17 05:22:51.520424 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:22:51.520392    2562 scope.go:117] "RemoveContainer" containerID="d4c4950c3164638d8d77bc03b5d90d9f2d54fcd5cb93584b12db36a271a07b95"
Jul 17 05:22:51.520815 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:22:51.520598    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm_odh-ai-gateway-infra(34c6ad80-2bd4-4768-94ee-c91a703f9f4d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d"
Jul 17 05:22:51.827604 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:22:51.827571    2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm"
Jul 17 05:22:52.525571 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:22:52.525544    2562 scope.go:117] "RemoveContainer" containerID="d4c4950c3164638d8d77bc03b5d90d9f2d54fcd5cb93584b12db36a271a07b95"
Jul 17 05:22:52.525976 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:22:52.525745    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm_odh-ai-gateway-infra(34c6ad80-2bd4-4768-94ee-c91a703f9f4d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d"
Jul 17 05:23:03.878464 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:23:03.878385    2562 scope.go:117] "RemoveContainer" containerID="d4c4950c3164638d8d77bc03b5d90d9f2d54fcd5cb93584b12db36a271a07b95"
Jul 17 05:23:03.878915 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:23:03.878560    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm_odh-ai-gateway-infra(34c6ad80-2bd4-4768-94ee-c91a703f9f4d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d"
Jul 17 05:23:14.878891 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:23:14.878857    2562 scope.go:117] "RemoveContainer" containerID="d4c4950c3164638d8d77bc03b5d90d9f2d54fcd5cb93584b12db36a271a07b95"
Jul 17 05:23:14.879556 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:23:14.879109    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm_odh-ai-gateway-infra(34c6ad80-2bd4-4768-94ee-c91a703f9f4d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d"
Jul 17 05:23:28.884013 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:23:28.883957    2562 scope.go:117] "RemoveContainer" containerID="d4c4950c3164638d8d77bc03b5d90d9f2d54fcd5cb93584b12db36a271a07b95"
Jul 17 05:23:28.884475 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:23:28.884174    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm_odh-ai-gateway-infra(34c6ad80-2bd4-4768-94ee-c91a703f9f4d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d"
Jul 17 05:23:40.878105 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:23:40.878068    2562 scope.go:117] "RemoveContainer" containerID="d4c4950c3164638d8d77bc03b5d90d9f2d54fcd5cb93584b12db36a271a07b95"
Jul 17 05:23:40.878678 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:23:40.878293    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm_odh-ai-gateway-infra(34c6ad80-2bd4-4768-94ee-c91a703f9f4d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d"
Jul 17 05:23:52.882504 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:23:52.882472    2562 scope.go:117] "RemoveContainer" containerID="d4c4950c3164638d8d77bc03b5d90d9f2d54fcd5cb93584b12db36a271a07b95"
Jul 17 05:23:52.883212 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:23:52.882692    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm_odh-ai-gateway-infra(34c6ad80-2bd4-4768-94ee-c91a703f9f4d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d"
Jul 17 05:24:03.878021 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:24:03.877957    2562 scope.go:117] "RemoveContainer" containerID="d4c4950c3164638d8d77bc03b5d90d9f2d54fcd5cb93584b12db36a271a07b95"
Jul 17 05:24:03.878517 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:24:03.878209    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm_odh-ai-gateway-infra(34c6ad80-2bd4-4768-94ee-c91a703f9f4d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d"
Jul 17 05:24:14.878389 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:24:14.878358    2562 scope.go:117] "RemoveContainer" containerID="d4c4950c3164638d8d77bc03b5d90d9f2d54fcd5cb93584b12db36a271a07b95"
Jul 17 05:24:15.939914 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:24:15.939879    2562 generic.go:356] "Generic (PLEG): container finished" podID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d" containerID="29cf6eb90548be9340d34f2bb9cc7340ce76cb64f4db2f4e1d2c547f2cd30f93" exitCode=1
Jul 17 05:24:15.940398 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:24:15.939925    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" event={"ID":"34c6ad80-2bd4-4768-94ee-c91a703f9f4d","Type":"ContainerDied","Data":"29cf6eb90548be9340d34f2bb9cc7340ce76cb64f4db2f4e1d2c547f2cd30f93"}
Jul 17 05:24:15.940398 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:24:15.940001    2562 scope.go:117] "RemoveContainer" containerID="d4c4950c3164638d8d77bc03b5d90d9f2d54fcd5cb93584b12db36a271a07b95"
Jul 17 05:24:15.940398 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:24:15.940295    2562 scope.go:117] "RemoveContainer" containerID="29cf6eb90548be9340d34f2bb9cc7340ce76cb64f4db2f4e1d2c547f2cd30f93"
Jul 17 05:24:15.940587 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:24:15.940505    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm_odh-ai-gateway-infra(34c6ad80-2bd4-4768-94ee-c91a703f9f4d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d"
Jul 17 05:24:19.506835 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:24:19.506799    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm"
Jul 17 05:24:19.507286 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:24:19.507254    2562 scope.go:117] "RemoveContainer" containerID="29cf6eb90548be9340d34f2bb9cc7340ce76cb64f4db2f4e1d2c547f2cd30f93"
Jul 17 05:24:19.507458 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:24:19.507436    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm_odh-ai-gateway-infra(34c6ad80-2bd4-4768-94ee-c91a703f9f4d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d"
Jul 17 05:24:21.827834 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:24:21.827791    2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm"
Jul 17 05:24:21.828308 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:24:21.828181    2562 scope.go:117] "RemoveContainer" containerID="29cf6eb90548be9340d34f2bb9cc7340ce76cb64f4db2f4e1d2c547f2cd30f93"
Jul 17 05:24:21.828368 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:24:21.828348    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm_odh-ai-gateway-infra(34c6ad80-2bd4-4768-94ee-c91a703f9f4d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d"
Jul 17 05:24:35.878718 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:24:35.878627    2562 scope.go:117] "RemoveContainer" containerID="29cf6eb90548be9340d34f2bb9cc7340ce76cb64f4db2f4e1d2c547f2cd30f93"
Jul 17 05:24:35.879259 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:24:35.878870    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm_odh-ai-gateway-infra(34c6ad80-2bd4-4768-94ee-c91a703f9f4d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d"
Jul 17 05:24:50.878809 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:24:50.878775    2562 scope.go:117] "RemoveContainer" containerID="29cf6eb90548be9340d34f2bb9cc7340ce76cb64f4db2f4e1d2c547f2cd30f93"
Jul 17 05:24:50.879400 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:24:50.878991    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm_odh-ai-gateway-infra(34c6ad80-2bd4-4768-94ee-c91a703f9f4d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d"
Jul 17 05:25:05.878209 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:25:05.878170    2562 scope.go:117] "RemoveContainer" containerID="29cf6eb90548be9340d34f2bb9cc7340ce76cb64f4db2f4e1d2c547f2cd30f93"
Jul 17 05:25:05.878624 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:25:05.878373    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm_odh-ai-gateway-infra(34c6ad80-2bd4-4768-94ee-c91a703f9f4d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d"
Jul 17 05:25:12.925875 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:25:12.925845    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tx9gj_c1baeee2-350c-43c5-9a04-fb34378edb5c/ovn-acl-logging/0.log"
Jul 17 05:25:12.927410 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:25:12.927375    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tx9gj_c1baeee2-350c-43c5-9a04-fb34378edb5c/ovn-acl-logging/0.log"
Jul 17 05:25:20.879194 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:25:20.879158    2562 scope.go:117] "RemoveContainer" containerID="29cf6eb90548be9340d34f2bb9cc7340ce76cb64f4db2f4e1d2c547f2cd30f93"
Jul 17 05:25:20.879568 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:25:20.879425    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm_odh-ai-gateway-infra(34c6ad80-2bd4-4768-94ee-c91a703f9f4d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d"
Jul 17 05:25:33.878059 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:25:33.878022    2562 scope.go:117] "RemoveContainer" containerID="29cf6eb90548be9340d34f2bb9cc7340ce76cb64f4db2f4e1d2c547f2cd30f93"
Jul 17 05:25:33.878564 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:25:33.878252    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm_odh-ai-gateway-infra(34c6ad80-2bd4-4768-94ee-c91a703f9f4d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d"
Jul 17 05:25:44.878630 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:25:44.878599    2562 scope.go:117] "RemoveContainer" containerID="29cf6eb90548be9340d34f2bb9cc7340ce76cb64f4db2f4e1d2c547f2cd30f93"
Jul 17 05:25:44.879120 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:25:44.878782    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm_odh-ai-gateway-infra(34c6ad80-2bd4-4768-94ee-c91a703f9f4d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d"
Jul 17 05:25:58.879145 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:25:58.879108    2562 scope.go:117] "RemoveContainer" containerID="29cf6eb90548be9340d34f2bb9cc7340ce76cb64f4db2f4e1d2c547f2cd30f93"
Jul 17 05:25:58.879610 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:25:58.879357    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm_odh-ai-gateway-infra(34c6ad80-2bd4-4768-94ee-c91a703f9f4d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d"
Jul 17 05:26:11.878002 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:26:11.877887    2562 scope.go:117] "RemoveContainer" containerID="29cf6eb90548be9340d34f2bb9cc7340ce76cb64f4db2f4e1d2c547f2cd30f93"
Jul 17 05:26:11.878465 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:26:11.878184    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm_odh-ai-gateway-infra(34c6ad80-2bd4-4768-94ee-c91a703f9f4d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d"
Jul 17 05:26:22.881428 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:26:22.881397    2562 scope.go:117] "RemoveContainer" containerID="29cf6eb90548be9340d34f2bb9cc7340ce76cb64f4db2f4e1d2c547f2cd30f93"
Jul 17 05:26:22.883894 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:26:22.881621    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm_odh-ai-gateway-infra(34c6ad80-2bd4-4768-94ee-c91a703f9f4d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d"
Jul 17 05:26:34.878122 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:26:34.878093    2562 scope.go:117] "RemoveContainer" containerID="29cf6eb90548be9340d34f2bb9cc7340ce76cb64f4db2f4e1d2c547f2cd30f93"
Jul 17 05:26:34.878590 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:26:34.878288    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm_odh-ai-gateway-infra(34c6ad80-2bd4-4768-94ee-c91a703f9f4d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d"
Jul 17 05:26:48.878386 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:26:48.878353    2562 scope.go:117] "RemoveContainer" containerID="29cf6eb90548be9340d34f2bb9cc7340ce76cb64f4db2f4e1d2c547f2cd30f93"
Jul 17 05:26:48.878772 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:26:48.878535    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm_odh-ai-gateway-infra(34c6ad80-2bd4-4768-94ee-c91a703f9f4d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d"
Jul 17 05:27:01.878057 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:01.878024    2562 scope.go:117] "RemoveContainer" containerID="29cf6eb90548be9340d34f2bb9cc7340ce76cb64f4db2f4e1d2c547f2cd30f93"
Jul 17 05:27:01.878484 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:01.878381    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 05:27:02.778080 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:02.778040    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" event={"ID":"34c6ad80-2bd4-4768-94ee-c91a703f9f4d","Type":"ContainerStarted","Data":"a61a59448900a0b05362cafa8c3b645ac75b014accaf50ca29ac9a588ea71a5d"}
Jul 17 05:27:02.778277 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:02.778259    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm"
Jul 17 05:27:03.784755 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:03.784719    2562 generic.go:356] "Generic (PLEG): container finished" podID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d" containerID="a61a59448900a0b05362cafa8c3b645ac75b014accaf50ca29ac9a588ea71a5d" exitCode=1
Jul 17 05:27:03.785248 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:03.784793    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" event={"ID":"34c6ad80-2bd4-4768-94ee-c91a703f9f4d","Type":"ContainerDied","Data":"a61a59448900a0b05362cafa8c3b645ac75b014accaf50ca29ac9a588ea71a5d"}
Jul 17 05:27:03.785248 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:03.784864    2562 scope.go:117] "RemoveContainer" containerID="29cf6eb90548be9340d34f2bb9cc7340ce76cb64f4db2f4e1d2c547f2cd30f93"
Jul 17 05:27:03.785248 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:03.785170    2562 scope.go:117] "RemoveContainer" containerID="a61a59448900a0b05362cafa8c3b645ac75b014accaf50ca29ac9a588ea71a5d"
Jul 17 05:27:03.785445 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:27:03.785421    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm_odh-ai-gateway-infra(34c6ad80-2bd4-4768-94ee-c91a703f9f4d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d"
Jul 17 05:27:04.792357 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:04.792324    2562 scope.go:117] "RemoveContainer" containerID="a61a59448900a0b05362cafa8c3b645ac75b014accaf50ca29ac9a588ea71a5d"
Jul 17 05:27:04.792750 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:27:04.792497    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm_odh-ai-gateway-infra(34c6ad80-2bd4-4768-94ee-c91a703f9f4d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d"
Jul 17 05:27:11.827679 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:11.827644    2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm"
Jul 17 05:27:11.828180 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:11.828040    2562 scope.go:117] "RemoveContainer" containerID="a61a59448900a0b05362cafa8c3b645ac75b014accaf50ca29ac9a588ea71a5d"
Jul 17 05:27:11.828234 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:27:11.828214    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm_odh-ai-gateway-infra(34c6ad80-2bd4-4768-94ee-c91a703f9f4d)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d"
Jul 17 05:27:16.756078 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:16.756034    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc"]
Jul 17 05:27:16.756500 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:16.756300    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc" podUID="cdb1b691-5f72-44e8-ae99-afcddb755ce3" containerName="istio-proxy" containerID="cri-o://bd9fb856d43a741919bea4356dd19ed1c88461ebf2d2c4e2f9b8e107ece8249b" gracePeriod=30
Jul 17 05:27:21.882562 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:21.882530    2562 generic.go:356] "Generic (PLEG): container finished" podID="cdb1b691-5f72-44e8-ae99-afcddb755ce3" containerID="bd9fb856d43a741919bea4356dd19ed1c88461ebf2d2c4e2f9b8e107ece8249b" exitCode=0
Jul 17 05:27:21.882892 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:21.882596    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc" event={"ID":"cdb1b691-5f72-44e8-ae99-afcddb755ce3","Type":"ContainerDied","Data":"bd9fb856d43a741919bea4356dd19ed1c88461ebf2d2c4e2f9b8e107ece8249b"}
Jul 17 05:27:21.882892 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:21.882666    2562 scope.go:117] "RemoveContainer" containerID="9f9758767f97f5baeb389a8103d5e3dc6c2bc7a146c7830259ca3b3f34a037d6"
Jul 17 05:27:22.023835 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.023772    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc"
Jul 17 05:27:22.068289 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.068251    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s979f\" (UniqueName: \"kubernetes.io/projected/cdb1b691-5f72-44e8-ae99-afcddb755ce3-kube-api-access-s979f\") pod \"cdb1b691-5f72-44e8-ae99-afcddb755ce3\" (UID: \"cdb1b691-5f72-44e8-ae99-afcddb755ce3\") "
Jul 17 05:27:22.068497 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.068405    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cdb1b691-5f72-44e8-ae99-afcddb755ce3-istio-token\") pod \"cdb1b691-5f72-44e8-ae99-afcddb755ce3\" (UID: \"cdb1b691-5f72-44e8-ae99-afcddb755ce3\") "
Jul 17 05:27:22.068497 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.068465    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cdb1b691-5f72-44e8-ae99-afcddb755ce3-istio-envoy\") pod \"cdb1b691-5f72-44e8-ae99-afcddb755ce3\" (UID: \"cdb1b691-5f72-44e8-ae99-afcddb755ce3\") "
Jul 17 05:27:22.068497 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.068490    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdb1b691-5f72-44e8-ae99-afcddb755ce3-credential-socket\") pod \"cdb1b691-5f72-44e8-ae99-afcddb755ce3\" (UID: \"cdb1b691-5f72-44e8-ae99-afcddb755ce3\") "
Jul 17 05:27:22.068674 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.068518    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cdb1b691-5f72-44e8-ae99-afcddb755ce3-istiod-ca-cert\") pod \"cdb1b691-5f72-44e8-ae99-afcddb755ce3\" (UID: \"cdb1b691-5f72-44e8-ae99-afcddb755ce3\") "
Jul 17 05:27:22.068674 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.068563    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cdb1b691-5f72-44e8-ae99-afcddb755ce3-istio-podinfo\") pod \"cdb1b691-5f72-44e8-ae99-afcddb755ce3\" (UID: \"cdb1b691-5f72-44e8-ae99-afcddb755ce3\") "
Jul 17 05:27:22.068674 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.068594    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cdb1b691-5f72-44e8-ae99-afcddb755ce3-workload-certs\") pod \"cdb1b691-5f72-44e8-ae99-afcddb755ce3\" (UID: \"cdb1b691-5f72-44e8-ae99-afcddb755ce3\") "
Jul 17 05:27:22.068674 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.068655    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdb1b691-5f72-44e8-ae99-afcddb755ce3-workload-socket\") pod \"cdb1b691-5f72-44e8-ae99-afcddb755ce3\" (UID: \"cdb1b691-5f72-44e8-ae99-afcddb755ce3\") "
Jul 17 05:27:22.068902 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.068680    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cdb1b691-5f72-44e8-ae99-afcddb755ce3-istio-data\") pod \"cdb1b691-5f72-44e8-ae99-afcddb755ce3\" (UID: \"cdb1b691-5f72-44e8-ae99-afcddb755ce3\") "
Jul 17 05:27:22.068902 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.068704    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdb1b691-5f72-44e8-ae99-afcddb755ce3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cdb1b691-5f72-44e8-ae99-afcddb755ce3" (UID: "cdb1b691-5f72-44e8-ae99-afcddb755ce3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:27:22.068902 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.068882    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdb1b691-5f72-44e8-ae99-afcddb755ce3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cdb1b691-5f72-44e8-ae99-afcddb755ce3" (UID: "cdb1b691-5f72-44e8-ae99-afcddb755ce3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:27:22.069102 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.068917    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdb1b691-5f72-44e8-ae99-afcddb755ce3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cdb1b691-5f72-44e8-ae99-afcddb755ce3" (UID: "cdb1b691-5f72-44e8-ae99-afcddb755ce3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:27:22.069102 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.069035    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdb1b691-5f72-44e8-ae99-afcddb755ce3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cdb1b691-5f72-44e8-ae99-afcddb755ce3" (UID: "cdb1b691-5f72-44e8-ae99-afcddb755ce3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:27:22.069215 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.069105    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdb1b691-5f72-44e8-ae99-afcddb755ce3-credential-socket\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:27:22.069215 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.069124    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cdb1b691-5f72-44e8-ae99-afcddb755ce3-istiod-ca-cert\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:27:22.069215 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.069139    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cdb1b691-5f72-44e8-ae99-afcddb755ce3-workload-certs\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:27:22.070076 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.070047    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdb1b691-5f72-44e8-ae99-afcddb755ce3-istio-data" (OuterVolumeSpecName: "istio-data") pod "cdb1b691-5f72-44e8-ae99-afcddb755ce3" (UID: "cdb1b691-5f72-44e8-ae99-afcddb755ce3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:27:22.070896 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.070866    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdb1b691-5f72-44e8-ae99-afcddb755ce3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cdb1b691-5f72-44e8-ae99-afcddb755ce3" (UID: "cdb1b691-5f72-44e8-ae99-afcddb755ce3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:27:22.071549 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.071520    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cdb1b691-5f72-44e8-ae99-afcddb755ce3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cdb1b691-5f72-44e8-ae99-afcddb755ce3" (UID: "cdb1b691-5f72-44e8-ae99-afcddb755ce3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 05:27:22.071747 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.071721    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdb1b691-5f72-44e8-ae99-afcddb755ce3-kube-api-access-s979f" (OuterVolumeSpecName: "kube-api-access-s979f") pod "cdb1b691-5f72-44e8-ae99-afcddb755ce3" (UID: "cdb1b691-5f72-44e8-ae99-afcddb755ce3"). InnerVolumeSpecName "kube-api-access-s979f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:27:22.071939 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.071917    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdb1b691-5f72-44e8-ae99-afcddb755ce3-istio-token" (OuterVolumeSpecName: "istio-token") pod "cdb1b691-5f72-44e8-ae99-afcddb755ce3" (UID: "cdb1b691-5f72-44e8-ae99-afcddb755ce3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:27:22.170274 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.170234    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s979f\" (UniqueName: \"kubernetes.io/projected/cdb1b691-5f72-44e8-ae99-afcddb755ce3-kube-api-access-s979f\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:27:22.170274 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.170267    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cdb1b691-5f72-44e8-ae99-afcddb755ce3-istio-token\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:27:22.170274 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.170276    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cdb1b691-5f72-44e8-ae99-afcddb755ce3-istio-envoy\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:27:22.170274 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.170284    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cdb1b691-5f72-44e8-ae99-afcddb755ce3-istio-podinfo\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:27:22.170539 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.170292    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdb1b691-5f72-44e8-ae99-afcddb755ce3-workload-socket\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:27:22.170539 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.170299    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cdb1b691-5f72-44e8-ae99-afcddb755ce3-istio-data\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:27:22.347896 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.347858    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm"]
Jul 17 05:27:22.479664 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.479635    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm"
Jul 17 05:27:22.574328 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.574296    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kbzl7\" (UniqueName: \"kubernetes.io/projected/34c6ad80-2bd4-4768-94ee-c91a703f9f4d-kube-api-access-kbzl7\") pod \"34c6ad80-2bd4-4768-94ee-c91a703f9f4d\" (UID: \"34c6ad80-2bd4-4768-94ee-c91a703f9f4d\") "
Jul 17 05:27:22.574515 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.574390    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/34c6ad80-2bd4-4768-94ee-c91a703f9f4d-openshift-service-ca\") pod \"34c6ad80-2bd4-4768-94ee-c91a703f9f4d\" (UID: \"34c6ad80-2bd4-4768-94ee-c91a703f9f4d\") "
Jul 17 05:27:22.574515 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.574507    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/34c6ad80-2bd4-4768-94ee-c91a703f9f4d-maas-api-tls\") pod \"34c6ad80-2bd4-4768-94ee-c91a703f9f4d\" (UID: \"34c6ad80-2bd4-4768-94ee-c91a703f9f4d\") "
Jul 17 05:27:22.574745 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.574710    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34c6ad80-2bd4-4768-94ee-c91a703f9f4d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "34c6ad80-2bd4-4768-94ee-c91a703f9f4d" (UID: "34c6ad80-2bd4-4768-94ee-c91a703f9f4d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:27:22.576497 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.576469    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/34c6ad80-2bd4-4768-94ee-c91a703f9f4d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "34c6ad80-2bd4-4768-94ee-c91a703f9f4d" (UID: "34c6ad80-2bd4-4768-94ee-c91a703f9f4d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 05:27:22.576622 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.576554    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34c6ad80-2bd4-4768-94ee-c91a703f9f4d-kube-api-access-kbzl7" (OuterVolumeSpecName: "kube-api-access-kbzl7") pod "34c6ad80-2bd4-4768-94ee-c91a703f9f4d" (UID: "34c6ad80-2bd4-4768-94ee-c91a703f9f4d"). InnerVolumeSpecName "kube-api-access-kbzl7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:27:22.675731 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.675643    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/34c6ad80-2bd4-4768-94ee-c91a703f9f4d-maas-api-tls\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:27:22.675731 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.675677    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kbzl7\" (UniqueName: \"kubernetes.io/projected/34c6ad80-2bd4-4768-94ee-c91a703f9f4d-kube-api-access-kbzl7\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:27:22.675731 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.675687    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/34c6ad80-2bd4-4768-94ee-c91a703f9f4d-openshift-service-ca\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:27:22.888810 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.888778    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm"
Jul 17 05:27:22.889298 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.888818    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm" event={"ID":"34c6ad80-2bd4-4768-94ee-c91a703f9f4d","Type":"ContainerDied","Data":"a12611038dc6153072dcc53087715f3cf17fde221159f5152f53e72800a677d8"}
Jul 17 05:27:22.889298 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.888882    2562 scope.go:117] "RemoveContainer" containerID="a61a59448900a0b05362cafa8c3b645ac75b014accaf50ca29ac9a588ea71a5d"
Jul 17 05:27:22.890638 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.890471    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc" event={"ID":"cdb1b691-5f72-44e8-ae99-afcddb755ce3","Type":"ContainerDied","Data":"a63160e76f60a0496e63db7ccb8c66a9bb0b5af5b5532a64e78c67ec94c64733"}
Jul 17 05:27:22.890638 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.890552    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc"
Jul 17 05:27:22.899829 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.899795    2562 scope.go:117] "RemoveContainer" containerID="bd9fb856d43a741919bea4356dd19ed1c88461ebf2d2c4e2f9b8e107ece8249b"
Jul 17 05:27:22.915638 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.915613    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc"]
Jul 17 05:27:22.919248 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.919228    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-9eb8486e-openshift-default-864db8bf5c-kl5tc"]
Jul 17 05:27:22.928623 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.928572    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm"]
Jul 17 05:27:22.932080 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:22.932059    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9eb8486e-cdb5bc8d9-kvtwm"]
Jul 17 05:27:24.885082 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:24.885048    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d" path="/var/lib/kubelet/pods/34c6ad80-2bd4-4768-94ee-c91a703f9f4d/volumes"
Jul 17 05:27:24.885558 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:24.885543    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cdb1b691-5f72-44e8-ae99-afcddb755ce3" path="/var/lib/kubelet/pods/cdb1b691-5f72-44e8-ae99-afcddb755ce3/volumes"
Jul 17 05:27:28.557520 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.557482    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8"]
Jul 17 05:27:28.558017 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.557881    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d" containerName="maas-api"
Jul 17 05:27:28.558017 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.557893    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d" containerName="maas-api"
Jul 17 05:27:28.558017 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.557906    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d" containerName="maas-api"
Jul 17 05:27:28.558017 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.557911    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d" containerName="maas-api"
Jul 17 05:27:28.558017 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.557918    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdb1b691-5f72-44e8-ae99-afcddb755ce3" containerName="istio-proxy"
Jul 17 05:27:28.558017 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.557924    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdb1b691-5f72-44e8-ae99-afcddb755ce3" containerName="istio-proxy"
Jul 17 05:27:28.558017 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.557930    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d" containerName="maas-api"
Jul 17 05:27:28.558017 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.557934    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d" containerName="maas-api"
Jul 17 05:27:28.558017 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.557951    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d" containerName="maas-api"
Jul 17 05:27:28.558017 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.557956    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d" containerName="maas-api"
Jul 17 05:27:28.558017 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.557989    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d" containerName="maas-api"
Jul 17 05:27:28.558017 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.557994    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d" containerName="maas-api"
Jul 17 05:27:28.558736 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.558055    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d" containerName="maas-api"
Jul 17 05:27:28.558736 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.558063    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d" containerName="maas-api"
Jul 17 05:27:28.558736 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.558069    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdb1b691-5f72-44e8-ae99-afcddb755ce3" containerName="istio-proxy"
Jul 17 05:27:28.558736 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.558075    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d" containerName="maas-api"
Jul 17 05:27:28.558736 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.558081    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d" containerName="maas-api"
Jul 17 05:27:28.558736 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.558086    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdb1b691-5f72-44e8-ae99-afcddb755ce3" containerName="istio-proxy"
Jul 17 05:27:28.558736 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.558091    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d" containerName="maas-api"
Jul 17 05:27:28.558736 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.558097    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d" containerName="maas-api"
Jul 17 05:27:28.558736 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.558104    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d" containerName="maas-api"
Jul 17 05:27:28.563140 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.563118    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8"
Jul 17 05:27:28.566267 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.566174    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-bf0ad9db-openshift-default-dockercfg-m8nvz\""
Jul 17 05:27:28.575226 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.575205    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8"]
Jul 17 05:27:28.629367 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.629332    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-istio-token\") pod \"e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8\" (UID: \"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364\") " pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8"
Jul 17 05:27:28.629367 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.629370    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hf2b2\" (UniqueName: \"kubernetes.io/projected/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-kube-api-access-hf2b2\") pod \"e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8\" (UID: \"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364\") " pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8"
Jul 17 05:27:28.629577 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.629419    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-workload-certs\") pod \"e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8\" (UID: \"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364\") " pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8"
Jul 17 05:27:28.629577 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.629436    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-istio-envoy\") pod \"e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8\" (UID: \"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364\") " pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8"
Jul 17 05:27:28.629577 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.629454    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-istio-podinfo\") pod \"e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8\" (UID: \"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364\") " pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8"
Jul 17 05:27:28.629577 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.629468    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-istiod-ca-cert\") pod \"e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8\" (UID: \"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364\") " pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8"
Jul 17 05:27:28.629792 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.629580    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-istio-data\") pod \"e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8\" (UID: \"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364\") " pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8"
Jul 17 05:27:28.629792 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.629637    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-credential-socket\") pod \"e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8\" (UID: \"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364\") " pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8"
Jul 17 05:27:28.629792 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.629671    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-workload-socket\") pod \"e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8\" (UID: \"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364\") " pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8"
Jul 17 05:27:28.731237 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.731205    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-istio-data\") pod \"e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8\" (UID: \"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364\") " pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8"
Jul 17 05:27:28.731421 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.731260    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-credential-socket\") pod \"e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8\" (UID: \"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364\") " pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8"
Jul 17 05:27:28.731421 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.731296    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-workload-socket\") pod \"e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8\" (UID: \"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364\") " pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8"
Jul 17 05:27:28.731421 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.731389    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-istio-token\") pod \"e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8\" (UID: \"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364\") " pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8"
Jul 17 05:27:28.731601 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.731418    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hf2b2\" (UniqueName: \"kubernetes.io/projected/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-kube-api-access-hf2b2\") pod \"e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8\" (UID: \"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364\") " pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8"
Jul 17 05:27:28.731601 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.731470    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-workload-certs\") pod \"e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8\" (UID: \"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364\") " pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8"
Jul 17 05:27:28.731601 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.731500    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-istio-envoy\") pod \"e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8\" (UID: \"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364\") " pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8"
Jul 17 05:27:28.731601 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.731527    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-istio-podinfo\") pod \"e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8\" (UID: \"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364\") " pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8"
Jul 17 05:27:28.731601 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.731548    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-istiod-ca-cert\") pod \"e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8\" (UID: \"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364\") " pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8"
Jul 17 05:27:28.731868 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.731635    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-istio-data\") pod \"e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8\" (UID: \"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364\") " pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8"
Jul 17 05:27:28.731868 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.731691    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-credential-socket\") pod \"e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8\" (UID: \"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364\") " pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8"
Jul 17 05:27:28.731868 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.731726    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-workload-socket\") pod \"e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8\" (UID: \"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364\") " pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8"
Jul 17 05:27:28.732140 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.731921    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-workload-certs\") pod \"e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8\" (UID: \"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364\") " pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8"
Jul 17 05:27:28.732320 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.732296    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-istiod-ca-cert\") pod \"e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8\" (UID: \"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364\") " pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8"
Jul 17 05:27:28.734250 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.734223    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-istio-envoy\") pod \"e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8\" (UID: \"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364\") " pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8"
Jul 17 05:27:28.734389 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.734370    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-istio-podinfo\") pod \"e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8\" (UID: \"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364\") " pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8"
Jul 17 05:27:28.739629 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.739606    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hf2b2\" (UniqueName: \"kubernetes.io/projected/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-kube-api-access-hf2b2\") pod \"e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8\" (UID: \"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364\") " pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8"
Jul 17 05:27:28.740652 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.740631    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-istio-token\") pod \"e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8\" (UID: \"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364\") " pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8"
Jul 17 05:27:28.875141 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:28.875100    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8"
Jul 17 05:27:29.014456 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:29.014419    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8"]
Jul 17 05:27:29.017406 ip-10-0-132-19 kubenswrapper[2562]: W0717 05:27:29.017380    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacf65c3a_a2db_4ecc_bc8e_fd9a1b72d364.slice/crio-8440203c008fa484da8fbbff6c190fde116c9ebc44cddb20354eb5708cab8af4 WatchSource:0}: Error finding container 8440203c008fa484da8fbbff6c190fde116c9ebc44cddb20354eb5708cab8af4: Status 404 returned error can't find the container with id 8440203c008fa484da8fbbff6c190fde116c9ebc44cddb20354eb5708cab8af4
Jul 17 05:27:29.019700 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:29.019662    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:27:29.019822 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:29.019730    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:27:29.019822 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:29.019760    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:27:29.780247 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:29.780206    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p"]
Jul 17 05:27:29.780915 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:29.780889    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdb1b691-5f72-44e8-ae99-afcddb755ce3" containerName="istio-proxy"
Jul 17 05:27:29.780915 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:29.780916    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdb1b691-5f72-44e8-ae99-afcddb755ce3" containerName="istio-proxy"
Jul 17 05:27:29.781121 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:29.780987    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d" containerName="maas-api"
Jul 17 05:27:29.781121 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:29.780999    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d" containerName="maas-api"
Jul 17 05:27:29.781121 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:29.781011    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d" containerName="maas-api"
Jul 17 05:27:29.781121 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:29.781021    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="34c6ad80-2bd4-4768-94ee-c91a703f9f4d" containerName="maas-api"
Jul 17 05:27:29.784781 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:29.784757    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p"
Jul 17 05:27:29.787547 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:29.787527    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-bf0ad9db-serving-cert\""
Jul 17 05:27:29.787547 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:29.787542    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-xwcd7\""
Jul 17 05:27:29.795524 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:29.795497    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p"]
Jul 17 05:27:29.842604 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:29.842568    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d09629f-8399-4354-ae12-785ee47c9b97-maas-api-tls\") pod \"maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p\" (UID: \"4d09629f-8399-4354-ae12-785ee47c9b97\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p"
Jul 17 05:27:29.842754 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:29.842711    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d09629f-8399-4354-ae12-785ee47c9b97-openshift-service-ca\") pod \"maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p\" (UID: \"4d09629f-8399-4354-ae12-785ee47c9b97\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p"
Jul 17 05:27:29.842754 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:29.842744    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cd47p\" (UniqueName: \"kubernetes.io/projected/4d09629f-8399-4354-ae12-785ee47c9b97-kube-api-access-cd47p\") pod \"maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p\" (UID: \"4d09629f-8399-4354-ae12-785ee47c9b97\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p"
Jul 17 05:27:29.929307 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:29.929272    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8" event={"ID":"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364","Type":"ContainerStarted","Data":"e84e6669c4d3b9f378f99d332b826d3682b3ccdf468d769d986b880b8e8ac7d1"}
Jul 17 05:27:29.929484 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:29.929325    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8" event={"ID":"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364","Type":"ContainerStarted","Data":"8440203c008fa484da8fbbff6c190fde116c9ebc44cddb20354eb5708cab8af4"}
Jul 17 05:27:29.943895 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:29.943862    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d09629f-8399-4354-ae12-785ee47c9b97-openshift-service-ca\") pod \"maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p\" (UID: \"4d09629f-8399-4354-ae12-785ee47c9b97\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p"
Jul 17 05:27:29.944072 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:29.943915    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cd47p\" (UniqueName: \"kubernetes.io/projected/4d09629f-8399-4354-ae12-785ee47c9b97-kube-api-access-cd47p\") pod \"maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p\" (UID: \"4d09629f-8399-4354-ae12-785ee47c9b97\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p"
Jul 17 05:27:29.944072 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:29.944028    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d09629f-8399-4354-ae12-785ee47c9b97-maas-api-tls\") pod \"maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p\" (UID: \"4d09629f-8399-4354-ae12-785ee47c9b97\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p"
Jul 17 05:27:29.948026 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:27:29.944619    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-serving-cert: secret "maas-api-e2e-derive-bf0ad9db-serving-cert" not found
Jul 17 05:27:29.948026 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:27:29.944773    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d09629f-8399-4354-ae12-785ee47c9b97-maas-api-tls podName:4d09629f-8399-4354-ae12-785ee47c9b97 nodeName:}" failed. No retries permitted until 2026-07-17 05:27:30.44473239 +0000 UTC m=+2238.128326024 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4d09629f-8399-4354-ae12-785ee47c9b97-maas-api-tls") pod "maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p" (UID: "4d09629f-8399-4354-ae12-785ee47c9b97") : secret "maas-api-e2e-derive-bf0ad9db-serving-cert" not found
Jul 17 05:27:29.948026 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:29.945172    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d09629f-8399-4354-ae12-785ee47c9b97-openshift-service-ca\") pod \"maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p\" (UID: \"4d09629f-8399-4354-ae12-785ee47c9b97\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p"
Jul 17 05:27:29.954076 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:29.954029    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8" podStartSLOduration=1.954012648 podStartE2EDuration="1.954012648s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:27:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:27:29.952496385 +0000 UTC m=+2237.636090041" watchObservedRunningTime="2026-07-17 05:27:29.954012648 +0000 UTC m=+2237.637606304"
Jul 17 05:27:29.955692 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:29.955661    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cd47p\" (UniqueName: \"kubernetes.io/projected/4d09629f-8399-4354-ae12-785ee47c9b97-kube-api-access-cd47p\") pod \"maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p\" (UID: \"4d09629f-8399-4354-ae12-785ee47c9b97\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p"
Jul 17 05:27:30.449657 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:30.449618    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d09629f-8399-4354-ae12-785ee47c9b97-maas-api-tls\") pod \"maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p\" (UID: \"4d09629f-8399-4354-ae12-785ee47c9b97\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p"
Jul 17 05:27:30.452431 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:30.452399    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d09629f-8399-4354-ae12-785ee47c9b97-maas-api-tls\") pod \"maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p\" (UID: \"4d09629f-8399-4354-ae12-785ee47c9b97\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p"
Jul 17 05:27:30.698235 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:30.698192    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p"
Jul 17 05:27:30.840001 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:30.839956    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p"]
Jul 17 05:27:30.842328 ip-10-0-132-19 kubenswrapper[2562]: W0717 05:27:30.842297    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d09629f_8399_4354_ae12_785ee47c9b97.slice/crio-f430434a43c09bdc48946128f55292b7366291870c9ba564b8b4aa575fe47b86 WatchSource:0}: Error finding container f430434a43c09bdc48946128f55292b7366291870c9ba564b8b4aa575fe47b86: Status 404 returned error can't find the container with id f430434a43c09bdc48946128f55292b7366291870c9ba564b8b4aa575fe47b86
Jul 17 05:27:30.875758 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:30.875728    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8"
Jul 17 05:27:30.884372 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:30.884350    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8"
Jul 17 05:27:30.935390 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:30.935353    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p" event={"ID":"4d09629f-8399-4354-ae12-785ee47c9b97","Type":"ContainerStarted","Data":"f430434a43c09bdc48946128f55292b7366291870c9ba564b8b4aa575fe47b86"}
Jul 17 05:27:30.935667 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:30.935641    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8"
Jul 17 05:27:30.936567 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:30.936547    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8"
Jul 17 05:27:31.941713 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:31.941617    2562 generic.go:356] "Generic (PLEG): container finished" podID="4d09629f-8399-4354-ae12-785ee47c9b97" containerID="13acc7bdb52836f19f16ef32a7bb8d85b96d7f3dc31fd6d80517c57437663dff" exitCode=1
Jul 17 05:27:31.942229 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:31.941703    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p" event={"ID":"4d09629f-8399-4354-ae12-785ee47c9b97","Type":"ContainerDied","Data":"13acc7bdb52836f19f16ef32a7bb8d85b96d7f3dc31fd6d80517c57437663dff"}
Jul 17 05:27:31.942229 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:31.941931    2562 scope.go:117] "RemoveContainer" containerID="13acc7bdb52836f19f16ef32a7bb8d85b96d7f3dc31fd6d80517c57437663dff"
Jul 17 05:27:32.948948 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:32.948874    2562 generic.go:356] "Generic (PLEG): container finished" podID="4d09629f-8399-4354-ae12-785ee47c9b97" containerID="197f7d010165771d27218dd79ea652475791d8cb6fd5d1e9dfc4b553ddd0682f" exitCode=1
Jul 17 05:27:32.949353 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:32.948981    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p" event={"ID":"4d09629f-8399-4354-ae12-785ee47c9b97","Type":"ContainerDied","Data":"197f7d010165771d27218dd79ea652475791d8cb6fd5d1e9dfc4b553ddd0682f"}
Jul 17 05:27:32.949353 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:32.949048    2562 scope.go:117] "RemoveContainer" containerID="13acc7bdb52836f19f16ef32a7bb8d85b96d7f3dc31fd6d80517c57437663dff"
Jul 17 05:27:32.949353 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:32.949287    2562 scope.go:117] "RemoveContainer" containerID="197f7d010165771d27218dd79ea652475791d8cb6fd5d1e9dfc4b553ddd0682f"
Jul 17 05:27:32.949545 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:27:32.949520    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p_odh-ai-gateway-infra(4d09629f-8399-4354-ae12-785ee47c9b97)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p" podUID="4d09629f-8399-4354-ae12-785ee47c9b97"
Jul 17 05:27:33.956791 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:33.956766    2562 scope.go:117] "RemoveContainer" containerID="197f7d010165771d27218dd79ea652475791d8cb6fd5d1e9dfc4b553ddd0682f"
Jul 17 05:27:33.957164 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:27:33.956983    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p_odh-ai-gateway-infra(4d09629f-8399-4354-ae12-785ee47c9b97)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p" podUID="4d09629f-8399-4354-ae12-785ee47c9b97"
Jul 17 05:27:35.699232 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:35.699185    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p"
Jul 17 05:27:35.699621 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:35.699587    2562 scope.go:117] "RemoveContainer" containerID="197f7d010165771d27218dd79ea652475791d8cb6fd5d1e9dfc4b553ddd0682f"
Jul 17 05:27:35.699813 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:27:35.699791    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p_odh-ai-gateway-infra(4d09629f-8399-4354-ae12-785ee47c9b97)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p" podUID="4d09629f-8399-4354-ae12-785ee47c9b97"
Jul 17 05:27:40.699153 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:40.699106    2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p"
Jul 17 05:27:40.699561 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:40.699546    2562 scope.go:117] "RemoveContainer" containerID="197f7d010165771d27218dd79ea652475791d8cb6fd5d1e9dfc4b553ddd0682f"
Jul 17 05:27:40.699742 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:27:40.699722    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p_odh-ai-gateway-infra(4d09629f-8399-4354-ae12-785ee47c9b97)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p" podUID="4d09629f-8399-4354-ae12-785ee47c9b97"
Jul 17 05:27:42.006544 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:42.006510    2562 generic.go:356] "Generic (PLEG): container finished" podID="acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364" containerID="e84e6669c4d3b9f378f99d332b826d3682b3ccdf468d769d986b880b8e8ac7d1" exitCode=0
Jul 17 05:27:42.006953 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:42.006583    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8" event={"ID":"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364","Type":"ContainerDied","Data":"e84e6669c4d3b9f378f99d332b826d3682b3ccdf468d769d986b880b8e8ac7d1"}
Jul 17 05:27:42.007034 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:42.006976    2562 scope.go:117] "RemoveContainer" containerID="e84e6669c4d3b9f378f99d332b826d3682b3ccdf468d769d986b880b8e8ac7d1"
Jul 17 05:27:42.008015 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:42.007987    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:27:42.008081 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:42.008053    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:27:42.008116 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:42.008085    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:27:43.013252 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:43.013216    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8" event={"ID":"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364","Type":"ContainerStarted","Data":"866f6c3b536b66025e957fb060c1b30747a7ff120de110448aedf602b7233722"}
Jul 17 05:27:43.875858 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:43.875818    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8"
Jul 17 05:27:43.877325 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:43.877292    2562 patch_prober.go:28] interesting pod/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 17 05:27:43.877476 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:43.877393    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8" podUID="acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 17 05:27:44.875920 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:44.875874    2562 patch_prober.go:28] interesting pod/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 17 05:27:44.876452 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:44.876005    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8" podUID="acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 17 05:27:45.876031 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:45.875986    2562 patch_prober.go:28] interesting pod/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 17 05:27:45.876531 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:45.876069    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8" podUID="acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 17 05:27:45.935753 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:45.935721    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8"
Jul 17 05:27:46.884650 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:46.884415    2562 patch_prober.go:28] interesting pod/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 17 05:27:46.884650 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:46.884475    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8" podUID="acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 17 05:27:47.879671 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:47.879641    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8"
Jul 17 05:27:47.880601 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:47.880582    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8"
Jul 17 05:27:53.878441 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:53.878409    2562 scope.go:117] "RemoveContainer" containerID="197f7d010165771d27218dd79ea652475791d8cb6fd5d1e9dfc4b553ddd0682f"
Jul 17 05:27:55.073560 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:55.073529    2562 generic.go:356] "Generic (PLEG): container finished" podID="4d09629f-8399-4354-ae12-785ee47c9b97" containerID="616f0bdedd0a61f8031e59ed662def6a09ff0a44cd36972d6db4559ead8cee98" exitCode=1
Jul 17 05:27:55.074058 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:55.073574    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p" event={"ID":"4d09629f-8399-4354-ae12-785ee47c9b97","Type":"ContainerDied","Data":"616f0bdedd0a61f8031e59ed662def6a09ff0a44cd36972d6db4559ead8cee98"}
Jul 17 05:27:55.074058 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:55.073624    2562 scope.go:117] "RemoveContainer" containerID="197f7d010165771d27218dd79ea652475791d8cb6fd5d1e9dfc4b553ddd0682f"
Jul 17 05:27:55.074058 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:55.074012    2562 scope.go:117] "RemoveContainer" containerID="616f0bdedd0a61f8031e59ed662def6a09ff0a44cd36972d6db4559ead8cee98"
Jul 17 05:27:55.074330 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:27:55.074305    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p_odh-ai-gateway-infra(4d09629f-8399-4354-ae12-785ee47c9b97)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p" podUID="4d09629f-8399-4354-ae12-785ee47c9b97"
Jul 17 05:27:55.698458 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:55.698414    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p"
Jul 17 05:27:56.080035 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:27:56.080006    2562 scope.go:117] "RemoveContainer" containerID="616f0bdedd0a61f8031e59ed662def6a09ff0a44cd36972d6db4559ead8cee98"
Jul 17 05:27:56.080450 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:27:56.080205    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p_odh-ai-gateway-infra(4d09629f-8399-4354-ae12-785ee47c9b97)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p" podUID="4d09629f-8399-4354-ae12-785ee47c9b97"
Jul 17 05:28:00.698335 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:28:00.698290    2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p"
Jul 17 05:28:00.698745 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:28:00.698716    2562 scope.go:117] "RemoveContainer" containerID="616f0bdedd0a61f8031e59ed662def6a09ff0a44cd36972d6db4559ead8cee98"
Jul 17 05:28:00.698914 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:28:00.698892    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p_odh-ai-gateway-infra(4d09629f-8399-4354-ae12-785ee47c9b97)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p" podUID="4d09629f-8399-4354-ae12-785ee47c9b97"
Jul 17 05:28:12.880895 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:28:12.880866    2562 scope.go:117] "RemoveContainer" containerID="616f0bdedd0a61f8031e59ed662def6a09ff0a44cd36972d6db4559ead8cee98"
Jul 17 05:28:12.881376 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:28:12.881075    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p_odh-ai-gateway-infra(4d09629f-8399-4354-ae12-785ee47c9b97)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p" podUID="4d09629f-8399-4354-ae12-785ee47c9b97"
Jul 17 05:28:25.877940 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:28:25.877907    2562 scope.go:117] "RemoveContainer" containerID="616f0bdedd0a61f8031e59ed662def6a09ff0a44cd36972d6db4559ead8cee98"
Jul 17 05:28:27.235559 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:28:27.235525    2562 generic.go:356] "Generic (PLEG): container finished" podID="4d09629f-8399-4354-ae12-785ee47c9b97" containerID="1adee27a6cf4ffb259fa28244faae066a9c285e7ff4e2774b1e1e9932f709893" exitCode=1
Jul 17 05:28:27.236048 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:28:27.235598    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p" event={"ID":"4d09629f-8399-4354-ae12-785ee47c9b97","Type":"ContainerDied","Data":"1adee27a6cf4ffb259fa28244faae066a9c285e7ff4e2774b1e1e9932f709893"}
Jul 17 05:28:27.236048 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:28:27.235655    2562 scope.go:117] "RemoveContainer" containerID="616f0bdedd0a61f8031e59ed662def6a09ff0a44cd36972d6db4559ead8cee98"
Jul 17 05:28:27.236178 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:28:27.236058    2562 scope.go:117] "RemoveContainer" containerID="1adee27a6cf4ffb259fa28244faae066a9c285e7ff4e2774b1e1e9932f709893"
Jul 17 05:28:27.236301 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:28:27.236279    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p_odh-ai-gateway-infra(4d09629f-8399-4354-ae12-785ee47c9b97)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p" podUID="4d09629f-8399-4354-ae12-785ee47c9b97"
Jul 17 05:28:30.698420 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:28:30.698386    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p"
Jul 17 05:28:30.698819 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:28:30.698469    2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p"
Jul 17 05:28:30.698860 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:28:30.698838    2562 scope.go:117] "RemoveContainer" containerID="1adee27a6cf4ffb259fa28244faae066a9c285e7ff4e2774b1e1e9932f709893"
Jul 17 05:28:30.699076 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:28:30.699054    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p_odh-ai-gateway-infra(4d09629f-8399-4354-ae12-785ee47c9b97)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p" podUID="4d09629f-8399-4354-ae12-785ee47c9b97"
Jul 17 05:28:43.878247 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:28:43.878216    2562 scope.go:117] "RemoveContainer" containerID="1adee27a6cf4ffb259fa28244faae066a9c285e7ff4e2774b1e1e9932f709893"
Jul 17 05:28:43.878687 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:28:43.878398    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p_odh-ai-gateway-infra(4d09629f-8399-4354-ae12-785ee47c9b97)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p" podUID="4d09629f-8399-4354-ae12-785ee47c9b97"
Jul 17 05:28:57.878423 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:28:57.878391    2562 scope.go:117] "RemoveContainer" containerID="1adee27a6cf4ffb259fa28244faae066a9c285e7ff4e2774b1e1e9932f709893"
Jul 17 05:28:57.878810 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:28:57.878605    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p_odh-ai-gateway-infra(4d09629f-8399-4354-ae12-785ee47c9b97)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p" podUID="4d09629f-8399-4354-ae12-785ee47c9b97"
Jul 17 05:29:12.880808 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:29:12.880736    2562 scope.go:117] "RemoveContainer" containerID="1adee27a6cf4ffb259fa28244faae066a9c285e7ff4e2774b1e1e9932f709893"
Jul 17 05:29:13.480489 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:29:13.480390    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p" event={"ID":"4d09629f-8399-4354-ae12-785ee47c9b97","Type":"ContainerStarted","Data":"b01358a95038102973adf0be67ad715a0a7988872f63f116b2363d3ec9bac20e"}
Jul 17 05:29:13.480688 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:29:13.480668    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p"
Jul 17 05:29:13.498123 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:29:13.498076    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p" podStartSLOduration=102.635041392 podStartE2EDuration="1m44.498060525s" totalImagesPullingTime="1.863019133s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:27:29 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:29:13.496867962 +0000 UTC m=+2341.180461652" watchObservedRunningTime="2026-07-17 05:29:13.498060525 +0000 UTC m=+2341.181654187"
Jul 17 05:29:14.486690 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:29:14.486656    2562 generic.go:356] "Generic (PLEG): container finished" podID="4d09629f-8399-4354-ae12-785ee47c9b97" containerID="b01358a95038102973adf0be67ad715a0a7988872f63f116b2363d3ec9bac20e" exitCode=1
Jul 17 05:29:14.487143 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:29:14.486746    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p" event={"ID":"4d09629f-8399-4354-ae12-785ee47c9b97","Type":"ContainerDied","Data":"b01358a95038102973adf0be67ad715a0a7988872f63f116b2363d3ec9bac20e"}
Jul 17 05:29:14.487143 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:29:14.486803    2562 scope.go:117] "RemoveContainer" containerID="1adee27a6cf4ffb259fa28244faae066a9c285e7ff4e2774b1e1e9932f709893"
Jul 17 05:29:14.487143 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:29:14.487082    2562 scope.go:117] "RemoveContainer" containerID="b01358a95038102973adf0be67ad715a0a7988872f63f116b2363d3ec9bac20e"
Jul 17 05:29:14.487331 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:29:14.487306    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p_odh-ai-gateway-infra(4d09629f-8399-4354-ae12-785ee47c9b97)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p" podUID="4d09629f-8399-4354-ae12-785ee47c9b97"
Jul 17 05:29:15.493411 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:29:15.493384    2562 scope.go:117] "RemoveContainer" containerID="b01358a95038102973adf0be67ad715a0a7988872f63f116b2363d3ec9bac20e"
Jul 17 05:29:15.493863 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:29:15.493559    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p_odh-ai-gateway-infra(4d09629f-8399-4354-ae12-785ee47c9b97)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p" podUID="4d09629f-8399-4354-ae12-785ee47c9b97"
Jul 17 05:29:20.699158 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:29:20.699123    2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p"
Jul 17 05:29:20.699642 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:29:20.699595    2562 scope.go:117] "RemoveContainer" containerID="b01358a95038102973adf0be67ad715a0a7988872f63f116b2363d3ec9bac20e"
Jul 17 05:29:20.699824 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:29:20.699799    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p_odh-ai-gateway-infra(4d09629f-8399-4354-ae12-785ee47c9b97)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p" podUID="4d09629f-8399-4354-ae12-785ee47c9b97"
Jul 17 05:29:32.887898 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:29:32.883589    2562 scope.go:117] "RemoveContainer" containerID="b01358a95038102973adf0be67ad715a0a7988872f63f116b2363d3ec9bac20e"
Jul 17 05:29:32.887898 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:29:32.883981    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p_odh-ai-gateway-infra(4d09629f-8399-4354-ae12-785ee47c9b97)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p" podUID="4d09629f-8399-4354-ae12-785ee47c9b97"
Jul 17 05:29:46.878082 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:29:46.878043    2562 scope.go:117] "RemoveContainer" containerID="b01358a95038102973adf0be67ad715a0a7988872f63f116b2363d3ec9bac20e"
Jul 17 05:29:46.878636 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:29:46.878317    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p_odh-ai-gateway-infra(4d09629f-8399-4354-ae12-785ee47c9b97)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p" podUID="4d09629f-8399-4354-ae12-785ee47c9b97"
Jul 17 05:29:58.878840 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:29:58.878808    2562 scope.go:117] "RemoveContainer" containerID="b01358a95038102973adf0be67ad715a0a7988872f63f116b2363d3ec9bac20e"
Jul 17 05:29:58.879262 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:29:58.879025    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p_odh-ai-gateway-infra(4d09629f-8399-4354-ae12-785ee47c9b97)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p" podUID="4d09629f-8399-4354-ae12-785ee47c9b97"
Jul 17 05:30:11.878870 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:11.878841    2562 scope.go:117] "RemoveContainer" containerID="b01358a95038102973adf0be67ad715a0a7988872f63f116b2363d3ec9bac20e"
Jul 17 05:30:11.879271 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:30:11.879028    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p_odh-ai-gateway-infra(4d09629f-8399-4354-ae12-785ee47c9b97)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p" podUID="4d09629f-8399-4354-ae12-785ee47c9b97"
Jul 17 05:30:12.960473 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:12.960448    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tx9gj_c1baeee2-350c-43c5-9a04-fb34378edb5c/ovn-acl-logging/0.log"
Jul 17 05:30:12.961916 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:12.961890    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tx9gj_c1baeee2-350c-43c5-9a04-fb34378edb5c/ovn-acl-logging/0.log"
Jul 17 05:30:23.878529 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:23.878495    2562 scope.go:117] "RemoveContainer" containerID="b01358a95038102973adf0be67ad715a0a7988872f63f116b2363d3ec9bac20e"
Jul 17 05:30:23.879139 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:30:23.878713    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p_odh-ai-gateway-infra(4d09629f-8399-4354-ae12-785ee47c9b97)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p" podUID="4d09629f-8399-4354-ae12-785ee47c9b97"
Jul 17 05:30:34.243149 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:34.243058    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8"]
Jul 17 05:30:34.243595 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:34.243296    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8" podUID="acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364" containerName="istio-proxy" containerID="cri-o://866f6c3b536b66025e957fb060c1b30747a7ff120de110448aedf602b7233722" gracePeriod=30
Jul 17 05:30:38.878878 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:38.878849    2562 scope.go:117] "RemoveContainer" containerID="b01358a95038102973adf0be67ad715a0a7988872f63f116b2363d3ec9bac20e"
Jul 17 05:30:39.493936 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:39.493909    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8"
Jul 17 05:30:39.571392 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:39.571362    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-workload-socket\") pod \"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364\" (UID: \"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364\") "
Jul 17 05:30:39.571586 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:39.571413    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-istio-envoy\") pod \"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364\" (UID: \"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364\") "
Jul 17 05:30:39.571586 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:39.571438    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-istio-podinfo\") pod \"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364\" (UID: \"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364\") "
Jul 17 05:30:39.571586 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:39.571488    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-istiod-ca-cert\") pod \"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364\" (UID: \"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364\") "
Jul 17 05:30:39.571586 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:39.571512    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-istio-data\") pod \"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364\" (UID: \"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364\") "
Jul 17 05:30:39.571586 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:39.571542    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-workload-certs\") pod \"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364\" (UID: \"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364\") "
Jul 17 05:30:39.571586 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:39.571576    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-istio-token\") pod \"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364\" (UID: \"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364\") "
Jul 17 05:30:39.571874 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:39.571606    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hf2b2\" (UniqueName: \"kubernetes.io/projected/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-kube-api-access-hf2b2\") pod \"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364\" (UID: \"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364\") "
Jul 17 05:30:39.571874 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:39.571666    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-credential-socket\") pod \"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364\" (UID: \"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364\") "
Jul 17 05:30:39.571874 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:39.571693    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364" (UID: "acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:30:39.572063 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:39.572047    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-workload-socket\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:30:39.572257 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:39.572237    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364" (UID: "acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:30:39.573630 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:39.573369    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-istio-data" (OuterVolumeSpecName: "istio-data") pod "acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364" (UID: "acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:30:39.573630 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:39.573447    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364" (UID: "acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:30:39.573630 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:39.573600    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364" (UID: "acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:30:39.574382 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:39.574347    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364" (UID: "acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:30:39.574490 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:39.574426    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-kube-api-access-hf2b2" (OuterVolumeSpecName: "kube-api-access-hf2b2") pod "acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364" (UID: "acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364"). InnerVolumeSpecName "kube-api-access-hf2b2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:30:39.574545 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:39.574525    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364" (UID: "acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 05:30:39.574616 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:39.574592    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-istio-token" (OuterVolumeSpecName: "istio-token") pod "acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364" (UID: "acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:30:39.672695 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:39.672660    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-istio-envoy\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:30:39.672695 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:39.672687    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-istio-podinfo\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:30:39.672695 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:39.672697    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-istiod-ca-cert\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:30:39.672695 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:39.672705    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-istio-data\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:30:39.672947 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:39.672713    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-workload-certs\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:30:39.672947 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:39.672721    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-istio-token\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:30:39.672947 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:39.672728    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hf2b2\" (UniqueName: \"kubernetes.io/projected/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-kube-api-access-hf2b2\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:30:39.672947 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:39.672737    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364-credential-socket\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:30:39.915440 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:39.915404    2562 generic.go:356] "Generic (PLEG): container finished" podID="acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364" containerID="866f6c3b536b66025e957fb060c1b30747a7ff120de110448aedf602b7233722" exitCode=0
Jul 17 05:30:39.915932 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:39.915453    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8"
Jul 17 05:30:39.915932 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:39.915489    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8" event={"ID":"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364","Type":"ContainerDied","Data":"866f6c3b536b66025e957fb060c1b30747a7ff120de110448aedf602b7233722"}
Jul 17 05:30:39.915932 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:39.915555    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8" event={"ID":"acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364","Type":"ContainerDied","Data":"8440203c008fa484da8fbbff6c190fde116c9ebc44cddb20354eb5708cab8af4"}
Jul 17 05:30:39.915932 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:39.915594    2562 scope.go:117] "RemoveContainer" containerID="866f6c3b536b66025e957fb060c1b30747a7ff120de110448aedf602b7233722"
Jul 17 05:30:39.917505 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:39.917480    2562 generic.go:356] "Generic (PLEG): container finished" podID="4d09629f-8399-4354-ae12-785ee47c9b97" containerID="c48015f9ac81928fa984c6669a6e6eb407bee3f86912eab4d177b74aedf08b55" exitCode=1
Jul 17 05:30:39.917618 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:39.917525    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p" event={"ID":"4d09629f-8399-4354-ae12-785ee47c9b97","Type":"ContainerDied","Data":"c48015f9ac81928fa984c6669a6e6eb407bee3f86912eab4d177b74aedf08b55"}
Jul 17 05:30:39.917842 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:39.917827    2562 scope.go:117] "RemoveContainer" containerID="c48015f9ac81928fa984c6669a6e6eb407bee3f86912eab4d177b74aedf08b55"
Jul 17 05:30:39.918064 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:30:39.918046    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p_odh-ai-gateway-infra(4d09629f-8399-4354-ae12-785ee47c9b97)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p" podUID="4d09629f-8399-4354-ae12-785ee47c9b97"
Jul 17 05:30:39.925878 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:39.925858    2562 scope.go:117] "RemoveContainer" containerID="e84e6669c4d3b9f378f99d332b826d3682b3ccdf468d769d986b880b8e8ac7d1"
Jul 17 05:30:39.934444 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:39.934419    2562 scope.go:117] "RemoveContainer" containerID="866f6c3b536b66025e957fb060c1b30747a7ff120de110448aedf602b7233722"
Jul 17 05:30:39.934704 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:30:39.934686    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"866f6c3b536b66025e957fb060c1b30747a7ff120de110448aedf602b7233722\": container with ID starting with 866f6c3b536b66025e957fb060c1b30747a7ff120de110448aedf602b7233722 not found: ID does not exist" containerID="866f6c3b536b66025e957fb060c1b30747a7ff120de110448aedf602b7233722"
Jul 17 05:30:39.934756 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:39.934714    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"866f6c3b536b66025e957fb060c1b30747a7ff120de110448aedf602b7233722"} err="failed to get container status \"866f6c3b536b66025e957fb060c1b30747a7ff120de110448aedf602b7233722\": rpc error: code = NotFound desc = could not find container \"866f6c3b536b66025e957fb060c1b30747a7ff120de110448aedf602b7233722\": container with ID starting with 866f6c3b536b66025e957fb060c1b30747a7ff120de110448aedf602b7233722 not found: ID does not exist"
Jul 17 05:30:39.934756 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:39.934732    2562 scope.go:117] "RemoveContainer" containerID="e84e6669c4d3b9f378f99d332b826d3682b3ccdf468d769d986b880b8e8ac7d1"
Jul 17 05:30:39.935023 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:30:39.935000    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e84e6669c4d3b9f378f99d332b826d3682b3ccdf468d769d986b880b8e8ac7d1\": container with ID starting with e84e6669c4d3b9f378f99d332b826d3682b3ccdf468d769d986b880b8e8ac7d1 not found: ID does not exist" containerID="e84e6669c4d3b9f378f99d332b826d3682b3ccdf468d769d986b880b8e8ac7d1"
Jul 17 05:30:39.935080 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:39.935027    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e84e6669c4d3b9f378f99d332b826d3682b3ccdf468d769d986b880b8e8ac7d1"} err="failed to get container status \"e84e6669c4d3b9f378f99d332b826d3682b3ccdf468d769d986b880b8e8ac7d1\": rpc error: code = NotFound desc = could not find container \"e84e6669c4d3b9f378f99d332b826d3682b3ccdf468d769d986b880b8e8ac7d1\": container with ID starting with e84e6669c4d3b9f378f99d332b826d3682b3ccdf468d769d986b880b8e8ac7d1 not found: ID does not exist"
Jul 17 05:30:39.935080 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:39.935040    2562 scope.go:117] "RemoveContainer" containerID="b01358a95038102973adf0be67ad715a0a7988872f63f116b2363d3ec9bac20e"
Jul 17 05:30:39.958204 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:39.958176    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p"]
Jul 17 05:30:39.964026 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:39.963988    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8"]
Jul 17 05:30:39.970619 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:39.970593    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-bf0ad9db-openshift-default-796cc778f9-4p8k8"]
Jul 17 05:30:40.076080 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:30:40.076049    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-serving-cert: secret "maas-api-e2e-derive-bf0ad9db-serving-cert" not found
Jul 17 05:30:40.076247 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:30:40.076129    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d09629f-8399-4354-ae12-785ee47c9b97-maas-api-tls podName:4d09629f-8399-4354-ae12-785ee47c9b97 nodeName:}" failed. No retries permitted until 2026-07-17 05:30:40.576111875 +0000 UTC m=+2428.259705509 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4d09629f-8399-4354-ae12-785ee47c9b97-maas-api-tls") pod "maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p" (UID: "4d09629f-8399-4354-ae12-785ee47c9b97") : secret "maas-api-e2e-derive-bf0ad9db-serving-cert" not found
Jul 17 05:30:40.582174 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:30:40.582136    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-serving-cert: secret "maas-api-e2e-derive-bf0ad9db-serving-cert" not found
Jul 17 05:30:40.582363 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:30:40.582222    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d09629f-8399-4354-ae12-785ee47c9b97-maas-api-tls podName:4d09629f-8399-4354-ae12-785ee47c9b97 nodeName:}" failed. No retries permitted until 2026-07-17 05:30:41.582200647 +0000 UTC m=+2429.265794283 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4d09629f-8399-4354-ae12-785ee47c9b97-maas-api-tls") pod "maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p" (UID: "4d09629f-8399-4354-ae12-785ee47c9b97") : secret "maas-api-e2e-derive-bf0ad9db-serving-cert" not found
Jul 17 05:30:40.698721 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:40.698683    2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p"
Jul 17 05:30:40.884265 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:40.884185    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364" path="/var/lib/kubelet/pods/acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364/volumes"
Jul 17 05:30:41.062972 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:41.062936    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p"
Jul 17 05:30:41.188645 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:41.188565    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d09629f-8399-4354-ae12-785ee47c9b97-maas-api-tls\") pod \"4d09629f-8399-4354-ae12-785ee47c9b97\" (UID: \"4d09629f-8399-4354-ae12-785ee47c9b97\") "
Jul 17 05:30:41.188645 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:41.188645    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d09629f-8399-4354-ae12-785ee47c9b97-openshift-service-ca\") pod \"4d09629f-8399-4354-ae12-785ee47c9b97\" (UID: \"4d09629f-8399-4354-ae12-785ee47c9b97\") "
Jul 17 05:30:41.188895 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:41.188687    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cd47p\" (UniqueName: \"kubernetes.io/projected/4d09629f-8399-4354-ae12-785ee47c9b97-kube-api-access-cd47p\") pod \"4d09629f-8399-4354-ae12-785ee47c9b97\" (UID: \"4d09629f-8399-4354-ae12-785ee47c9b97\") "
Jul 17 05:30:41.189050 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:41.189028    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d09629f-8399-4354-ae12-785ee47c9b97-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4d09629f-8399-4354-ae12-785ee47c9b97" (UID: "4d09629f-8399-4354-ae12-785ee47c9b97"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:30:41.190842 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:41.190814    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d09629f-8399-4354-ae12-785ee47c9b97-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4d09629f-8399-4354-ae12-785ee47c9b97" (UID: "4d09629f-8399-4354-ae12-785ee47c9b97"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 05:30:41.190944 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:41.190883    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d09629f-8399-4354-ae12-785ee47c9b97-kube-api-access-cd47p" (OuterVolumeSpecName: "kube-api-access-cd47p") pod "4d09629f-8399-4354-ae12-785ee47c9b97" (UID: "4d09629f-8399-4354-ae12-785ee47c9b97"). InnerVolumeSpecName "kube-api-access-cd47p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:30:41.289866 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:41.289832    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d09629f-8399-4354-ae12-785ee47c9b97-maas-api-tls\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:30:41.289866 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:41.289860    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d09629f-8399-4354-ae12-785ee47c9b97-openshift-service-ca\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:30:41.289866 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:41.289870    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cd47p\" (UniqueName: \"kubernetes.io/projected/4d09629f-8399-4354-ae12-785ee47c9b97-kube-api-access-cd47p\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:30:41.929982 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:41.929940    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p"
Jul 17 05:30:41.929982 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:41.929956    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p" event={"ID":"4d09629f-8399-4354-ae12-785ee47c9b97","Type":"ContainerDied","Data":"f430434a43c09bdc48946128f55292b7366291870c9ba564b8b4aa575fe47b86"}
Jul 17 05:30:41.930237 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:41.930028    2562 scope.go:117] "RemoveContainer" containerID="c48015f9ac81928fa984c6669a6e6eb407bee3f86912eab4d177b74aedf08b55"
Jul 17 05:30:41.955331 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:41.955306    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p"]
Jul 17 05:30:41.963354 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:41.963333    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-bf0ad9db-c74747bf5-fmq4p"]
Jul 17 05:30:42.884526 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:30:42.884491    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d09629f-8399-4354-ae12-785ee47c9b97" path="/var/lib/kubelet/pods/4d09629f-8399-4354-ae12-785ee47c9b97/volumes"
Jul 17 05:31:04.307808 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:04.307766    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-wlcm4"]
Jul 17 05:31:04.308381 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:04.308358    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364" containerName="istio-proxy"
Jul 17 05:31:04.308381 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:04.308379    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364" containerName="istio-proxy"
Jul 17 05:31:04.308513 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:04.308411    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d09629f-8399-4354-ae12-785ee47c9b97" containerName="maas-api"
Jul 17 05:31:04.308513 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:04.308420    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d09629f-8399-4354-ae12-785ee47c9b97" containerName="maas-api"
Jul 17 05:31:04.308513 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:04.308434    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d09629f-8399-4354-ae12-785ee47c9b97" containerName="maas-api"
Jul 17 05:31:04.308513 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:04.308442    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d09629f-8399-4354-ae12-785ee47c9b97" containerName="maas-api"
Jul 17 05:31:04.308513 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:04.308450    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d09629f-8399-4354-ae12-785ee47c9b97" containerName="maas-api"
Jul 17 05:31:04.308513 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:04.308458    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d09629f-8399-4354-ae12-785ee47c9b97" containerName="maas-api"
Jul 17 05:31:04.308513 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:04.308487    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d09629f-8399-4354-ae12-785ee47c9b97" containerName="maas-api"
Jul 17 05:31:04.308513 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:04.308495    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d09629f-8399-4354-ae12-785ee47c9b97" containerName="maas-api"
Jul 17 05:31:04.308906 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:04.308578    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d09629f-8399-4354-ae12-785ee47c9b97" containerName="maas-api"
Jul 17 05:31:04.308906 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:04.308589    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d09629f-8399-4354-ae12-785ee47c9b97" containerName="maas-api"
Jul 17 05:31:04.308906 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:04.308598    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364" containerName="istio-proxy"
Jul 17 05:31:04.308906 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:04.308606    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d09629f-8399-4354-ae12-785ee47c9b97" containerName="maas-api"
Jul 17 05:31:04.308906 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:04.308617    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d09629f-8399-4354-ae12-785ee47c9b97" containerName="maas-api"
Jul 17 05:31:04.308906 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:04.308629    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364" containerName="istio-proxy"
Jul 17 05:31:04.314459 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:04.314435    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-wlcm4"
Jul 17 05:31:04.317477 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:04.317451    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qh8zj\""
Jul 17 05:31:04.318300 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:04.318275    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-wlcm4"]
Jul 17 05:31:04.404821 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:04.404781    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5013f41a-a853-426d-a17e-0149d2ce7836-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-bf0ad9db-wlcm4\" (UID: \"5013f41a-a853-426d-a17e-0149d2ce7836\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-wlcm4"
Jul 17 05:31:04.506000 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:04.505924    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5013f41a-a853-426d-a17e-0149d2ce7836-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-bf0ad9db-wlcm4\" (UID: \"5013f41a-a853-426d-a17e-0149d2ce7836\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-wlcm4"
Jul 17 05:31:04.506590 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:04.506568    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5013f41a-a853-426d-a17e-0149d2ce7836-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-bf0ad9db-wlcm4\" (UID: \"5013f41a-a853-426d-a17e-0149d2ce7836\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-wlcm4"
Jul 17 05:31:04.626933 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:04.626897    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-wlcm4"
Jul 17 05:31:04.762224 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:04.762201    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-wlcm4"]
Jul 17 05:31:04.764874 ip-10-0-132-19 kubenswrapper[2562]: W0717 05:31:04.764846    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5013f41a_a853_426d_a17e_0149d2ce7836.slice/crio-a32c232629294b28ebd86c007bb44c8d114b9f077bbb6849100e3a4fbfcfb7ae WatchSource:0}: Error finding container a32c232629294b28ebd86c007bb44c8d114b9f077bbb6849100e3a4fbfcfb7ae: Status 404 returned error can't find the container with id a32c232629294b28ebd86c007bb44c8d114b9f077bbb6849100e3a4fbfcfb7ae
Jul 17 05:31:05.047668 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:05.047627    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-wlcm4" event={"ID":"5013f41a-a853-426d-a17e-0149d2ce7836","Type":"ContainerStarted","Data":"a32c232629294b28ebd86c007bb44c8d114b9f077bbb6849100e3a4fbfcfb7ae"}
Jul 17 05:31:06.053751 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:06.053711    2562 generic.go:356] "Generic (PLEG): container finished" podID="5013f41a-a853-426d-a17e-0149d2ce7836" containerID="81c0527b47dd8d2676e523eec6ce8610551f2f2dbd4b897ec92e7ea4d5e78838" exitCode=6
Jul 17 05:31:06.054183 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:06.053801    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-wlcm4" event={"ID":"5013f41a-a853-426d-a17e-0149d2ce7836","Type":"ContainerDied","Data":"81c0527b47dd8d2676e523eec6ce8610551f2f2dbd4b897ec92e7ea4d5e78838"}
Jul 17 05:31:06.054183 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:06.053956    2562 scope.go:117] "RemoveContainer" containerID="81c0527b47dd8d2676e523eec6ce8610551f2f2dbd4b897ec92e7ea4d5e78838"
Jul 17 05:31:07.060670 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:07.060586    2562 generic.go:356] "Generic (PLEG): container finished" podID="5013f41a-a853-426d-a17e-0149d2ce7836" containerID="4d67c8ec8237f169ca374e15baf50314c714c880c75200c9209ee4cae0f15803" exitCode=6
Jul 17 05:31:07.060670 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:07.060627    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-wlcm4" event={"ID":"5013f41a-a853-426d-a17e-0149d2ce7836","Type":"ContainerDied","Data":"4d67c8ec8237f169ca374e15baf50314c714c880c75200c9209ee4cae0f15803"}
Jul 17 05:31:07.061148 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:07.060672    2562 scope.go:117] "RemoveContainer" containerID="81c0527b47dd8d2676e523eec6ce8610551f2f2dbd4b897ec92e7ea4d5e78838"
Jul 17 05:31:07.061148 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:07.060888    2562 scope.go:117] "RemoveContainer" containerID="4d67c8ec8237f169ca374e15baf50314c714c880c75200c9209ee4cae0f15803"
Jul 17 05:31:07.061228 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:31:07.061161    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-bf0ad9db-wlcm4_odh-ai-gateway-infra(5013f41a-a853-426d-a17e-0149d2ce7836)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-wlcm4" podUID="5013f41a-a853-426d-a17e-0149d2ce7836"
Jul 17 05:31:08.067221 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:08.067193    2562 scope.go:117] "RemoveContainer" containerID="4d67c8ec8237f169ca374e15baf50314c714c880c75200c9209ee4cae0f15803"
Jul 17 05:31:08.067607 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:31:08.067385    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-bf0ad9db-wlcm4_odh-ai-gateway-infra(5013f41a-a853-426d-a17e-0149d2ce7836)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-wlcm4" podUID="5013f41a-a853-426d-a17e-0149d2ce7836"
Jul 17 05:31:10.787797 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:10.787762    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-h4d4l"]
Jul 17 05:31:10.788237 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:10.788216    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d09629f-8399-4354-ae12-785ee47c9b97" containerName="maas-api"
Jul 17 05:31:10.788237 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:10.788229    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d09629f-8399-4354-ae12-785ee47c9b97" containerName="maas-api"
Jul 17 05:31:10.788327 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:10.788251    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364" containerName="istio-proxy"
Jul 17 05:31:10.788327 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:10.788257    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="acf65c3a-a2db-4ecc-bc8e-fd9a1b72d364" containerName="istio-proxy"
Jul 17 05:31:10.788391 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:10.788339    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d09629f-8399-4354-ae12-785ee47c9b97" containerName="maas-api"
Jul 17 05:31:10.788391 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:10.788347    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d09629f-8399-4354-ae12-785ee47c9b97" containerName="maas-api"
Jul 17 05:31:10.792476 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:10.792460    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-h4d4l"
Jul 17 05:31:10.798605 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:10.798565    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-h4d4l"]
Jul 17 05:31:10.867937 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:10.867903    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15862f31-2950-4f3d-bf65-2473c0a25f96-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4e221674-h4d4l\" (UID: \"15862f31-2950-4f3d-bf65-2473c0a25f96\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-h4d4l"
Jul 17 05:31:10.968579 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:10.968539    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15862f31-2950-4f3d-bf65-2473c0a25f96-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4e221674-h4d4l\" (UID: \"15862f31-2950-4f3d-bf65-2473c0a25f96\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-h4d4l"
Jul 17 05:31:10.969270 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:10.969243    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15862f31-2950-4f3d-bf65-2473c0a25f96-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4e221674-h4d4l\" (UID: \"15862f31-2950-4f3d-bf65-2473c0a25f96\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-h4d4l"
Jul 17 05:31:11.105457 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:11.105421    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-h4d4l"
Jul 17 05:31:11.240415 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:11.240391    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-h4d4l"]
Jul 17 05:31:11.242594 ip-10-0-132-19 kubenswrapper[2562]: W0717 05:31:11.242566    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15862f31_2950_4f3d_bf65_2473c0a25f96.slice/crio-6c85f993335b89b33a1a1b495b315d762d8e2d0801e08706dd2d418263700123 WatchSource:0}: Error finding container 6c85f993335b89b33a1a1b495b315d762d8e2d0801e08706dd2d418263700123: Status 404 returned error can't find the container with id 6c85f993335b89b33a1a1b495b315d762d8e2d0801e08706dd2d418263700123
Jul 17 05:31:12.088291 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:12.088255    2562 generic.go:356] "Generic (PLEG): container finished" podID="15862f31-2950-4f3d-bf65-2473c0a25f96" containerID="70b29350115c287fdb2879b2ebb12b6cc01c14b70538cb017228403f5a07a343" exitCode=6
Jul 17 05:31:12.088701 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:12.088372    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-h4d4l" event={"ID":"15862f31-2950-4f3d-bf65-2473c0a25f96","Type":"ContainerDied","Data":"70b29350115c287fdb2879b2ebb12b6cc01c14b70538cb017228403f5a07a343"}
Jul 17 05:31:12.088701 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:12.088422    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-h4d4l" event={"ID":"15862f31-2950-4f3d-bf65-2473c0a25f96","Type":"ContainerStarted","Data":"6c85f993335b89b33a1a1b495b315d762d8e2d0801e08706dd2d418263700123"}
Jul 17 05:31:12.088701 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:12.088504    2562 scope.go:117] "RemoveContainer" containerID="70b29350115c287fdb2879b2ebb12b6cc01c14b70538cb017228403f5a07a343"
Jul 17 05:31:13.098020 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:13.097985    2562 generic.go:356] "Generic (PLEG): container finished" podID="15862f31-2950-4f3d-bf65-2473c0a25f96" containerID="236a9a1c3c79dbfd50a90813075dec1fd3432c2d357176a432e452fb6dbe4ca6" exitCode=6
Jul 17 05:31:13.098470 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:13.098025    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-h4d4l" event={"ID":"15862f31-2950-4f3d-bf65-2473c0a25f96","Type":"ContainerDied","Data":"236a9a1c3c79dbfd50a90813075dec1fd3432c2d357176a432e452fb6dbe4ca6"}
Jul 17 05:31:13.098470 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:13.098075    2562 scope.go:117] "RemoveContainer" containerID="70b29350115c287fdb2879b2ebb12b6cc01c14b70538cb017228403f5a07a343"
Jul 17 05:31:13.098470 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:13.098255    2562 scope.go:117] "RemoveContainer" containerID="236a9a1c3c79dbfd50a90813075dec1fd3432c2d357176a432e452fb6dbe4ca6"
Jul 17 05:31:13.098603 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:31:13.098487    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-4e221674-h4d4l_odh-ai-gateway-infra(15862f31-2950-4f3d-bf65-2473c0a25f96)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-h4d4l" podUID="15862f31-2950-4f3d-bf65-2473c0a25f96"
Jul 17 05:31:14.104207 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:14.104179    2562 scope.go:117] "RemoveContainer" containerID="236a9a1c3c79dbfd50a90813075dec1fd3432c2d357176a432e452fb6dbe4ca6"
Jul 17 05:31:14.104593 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:31:14.104355    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-4e221674-h4d4l_odh-ai-gateway-infra(15862f31-2950-4f3d-bf65-2473c0a25f96)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-h4d4l" podUID="15862f31-2950-4f3d-bf65-2473c0a25f96"
Jul 17 05:31:23.878880 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:23.878844    2562 scope.go:117] "RemoveContainer" containerID="4d67c8ec8237f169ca374e15baf50314c714c880c75200c9209ee4cae0f15803"
Jul 17 05:31:24.156037 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:24.155935    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-wlcm4" event={"ID":"5013f41a-a853-426d-a17e-0149d2ce7836","Type":"ContainerStarted","Data":"4b5b0d3f4b6b18dbf36f49af67bcf3e213c781c09366e348ede6c6cd232fbfa7"}
Jul 17 05:31:24.172542 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:24.172492    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-wlcm4" podStartSLOduration=20.172473593 podStartE2EDuration="20.172473593s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:31:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:31:24.171935226 +0000 UTC m=+2471.855528904" watchObservedRunningTime="2026-07-17 05:31:24.172473593 +0000 UTC m=+2471.856067254"
Jul 17 05:31:25.162098 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:25.162063    2562 generic.go:356] "Generic (PLEG): container finished" podID="5013f41a-a853-426d-a17e-0149d2ce7836" containerID="4b5b0d3f4b6b18dbf36f49af67bcf3e213c781c09366e348ede6c6cd232fbfa7" exitCode=6
Jul 17 05:31:25.162584 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:25.162115    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-wlcm4" event={"ID":"5013f41a-a853-426d-a17e-0149d2ce7836","Type":"ContainerDied","Data":"4b5b0d3f4b6b18dbf36f49af67bcf3e213c781c09366e348ede6c6cd232fbfa7"}
Jul 17 05:31:25.162584 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:25.162178    2562 scope.go:117] "RemoveContainer" containerID="4d67c8ec8237f169ca374e15baf50314c714c880c75200c9209ee4cae0f15803"
Jul 17 05:31:25.162584 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:25.162436    2562 scope.go:117] "RemoveContainer" containerID="4b5b0d3f4b6b18dbf36f49af67bcf3e213c781c09366e348ede6c6cd232fbfa7"
Jul 17 05:31:25.162751 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:31:25.162728    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-bf0ad9db-wlcm4_odh-ai-gateway-infra(5013f41a-a853-426d-a17e-0149d2ce7836)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-wlcm4" podUID="5013f41a-a853-426d-a17e-0149d2ce7836"
Jul 17 05:31:25.183130 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:25.183103    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-wlcm4"]
Jul 17 05:31:26.309324 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:26.309302    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-wlcm4"
Jul 17 05:31:26.419325 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:26.419239    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5013f41a-a853-426d-a17e-0149d2ce7836-maas-api-service-ca\") pod \"5013f41a-a853-426d-a17e-0149d2ce7836\" (UID: \"5013f41a-a853-426d-a17e-0149d2ce7836\") "
Jul 17 05:31:26.419607 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:26.419583    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5013f41a-a853-426d-a17e-0149d2ce7836-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5013f41a-a853-426d-a17e-0149d2ce7836" (UID: "5013f41a-a853-426d-a17e-0149d2ce7836"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:31:26.520531 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:26.520494    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5013f41a-a853-426d-a17e-0149d2ce7836-maas-api-service-ca\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:31:27.174305 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:27.174214    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-wlcm4"
Jul 17 05:31:27.174305 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:27.174215    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-wlcm4" event={"ID":"5013f41a-a853-426d-a17e-0149d2ce7836","Type":"ContainerDied","Data":"a32c232629294b28ebd86c007bb44c8d114b9f077bbb6849100e3a4fbfcfb7ae"}
Jul 17 05:31:27.174517 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:27.174343    2562 scope.go:117] "RemoveContainer" containerID="4b5b0d3f4b6b18dbf36f49af67bcf3e213c781c09366e348ede6c6cd232fbfa7"
Jul 17 05:31:27.191880 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:27.191840    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-wlcm4"]
Jul 17 05:31:27.194032 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:27.194008    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-wlcm4"]
Jul 17 05:31:27.878892 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:27.878861    2562 scope.go:117] "RemoveContainer" containerID="236a9a1c3c79dbfd50a90813075dec1fd3432c2d357176a432e452fb6dbe4ca6"
Jul 17 05:31:28.181487 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:28.181399    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-h4d4l" event={"ID":"15862f31-2950-4f3d-bf65-2473c0a25f96","Type":"ContainerStarted","Data":"973600467e23268c6502e818ad2d15bf00cc801b1671c4bd271a34e60f4c0c47"}
Jul 17 05:31:28.198644 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:28.198595    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-h4d4l" podStartSLOduration=18.198579221 podStartE2EDuration="18.198579221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:31:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:31:28.197567532 +0000 UTC m=+2475.881161202" watchObservedRunningTime="2026-07-17 05:31:28.198579221 +0000 UTC m=+2475.882172877"
Jul 17 05:31:28.883858 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:28.883821    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5013f41a-a853-426d-a17e-0149d2ce7836" path="/var/lib/kubelet/pods/5013f41a-a853-426d-a17e-0149d2ce7836/volumes"
Jul 17 05:31:29.188794 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:29.188705    2562 generic.go:356] "Generic (PLEG): container finished" podID="15862f31-2950-4f3d-bf65-2473c0a25f96" containerID="973600467e23268c6502e818ad2d15bf00cc801b1671c4bd271a34e60f4c0c47" exitCode=6
Jul 17 05:31:29.188794 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:29.188751    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-h4d4l" event={"ID":"15862f31-2950-4f3d-bf65-2473c0a25f96","Type":"ContainerDied","Data":"973600467e23268c6502e818ad2d15bf00cc801b1671c4bd271a34e60f4c0c47"}
Jul 17 05:31:29.189034 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:29.188800    2562 scope.go:117] "RemoveContainer" containerID="236a9a1c3c79dbfd50a90813075dec1fd3432c2d357176a432e452fb6dbe4ca6"
Jul 17 05:31:29.189134 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:29.189115    2562 scope.go:117] "RemoveContainer" containerID="973600467e23268c6502e818ad2d15bf00cc801b1671c4bd271a34e60f4c0c47"
Jul 17 05:31:29.189348 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:31:29.189328    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-4e221674-h4d4l_odh-ai-gateway-infra(15862f31-2950-4f3d-bf65-2473c0a25f96)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-h4d4l" podUID="15862f31-2950-4f3d-bf65-2473c0a25f96"
Jul 17 05:31:29.213077 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:29.213048    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-h4d4l"]
Jul 17 05:31:30.339349 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:30.339323    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-h4d4l"
Jul 17 05:31:30.459529 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:30.459451    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15862f31-2950-4f3d-bf65-2473c0a25f96-maas-api-service-ca\") pod \"15862f31-2950-4f3d-bf65-2473c0a25f96\" (UID: \"15862f31-2950-4f3d-bf65-2473c0a25f96\") "
Jul 17 05:31:30.459790 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:30.459766    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15862f31-2950-4f3d-bf65-2473c0a25f96-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "15862f31-2950-4f3d-bf65-2473c0a25f96" (UID: "15862f31-2950-4f3d-bf65-2473c0a25f96"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:31:30.560267 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:30.560235    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15862f31-2950-4f3d-bf65-2473c0a25f96-maas-api-service-ca\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:31:31.201549 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:31.201469    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-h4d4l"
Jul 17 05:31:31.201549 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:31.201508    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-h4d4l" event={"ID":"15862f31-2950-4f3d-bf65-2473c0a25f96","Type":"ContainerDied","Data":"6c85f993335b89b33a1a1b495b315d762d8e2d0801e08706dd2d418263700123"}
Jul 17 05:31:31.201796 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:31.201572    2562 scope.go:117] "RemoveContainer" containerID="973600467e23268c6502e818ad2d15bf00cc801b1671c4bd271a34e60f4c0c47"
Jul 17 05:31:31.218141 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:31.218115    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-h4d4l"]
Jul 17 05:31:31.225594 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:31.225569    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-h4d4l"]
Jul 17 05:31:32.890531 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:32.890489    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15862f31-2950-4f3d-bf65-2473c0a25f96" path="/var/lib/kubelet/pods/15862f31-2950-4f3d-bf65-2473c0a25f96/volumes"
Jul 17 05:31:37.009326 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:37.009290    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-tlwwq"]
Jul 17 05:31:37.009718 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:37.009699    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15862f31-2950-4f3d-bf65-2473c0a25f96" containerName="revoke-keys"
Jul 17 05:31:37.009718 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:37.009711    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="15862f31-2950-4f3d-bf65-2473c0a25f96" containerName="revoke-keys"
Jul 17 05:31:37.009791 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:37.009720    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d09629f-8399-4354-ae12-785ee47c9b97" containerName="maas-api"
Jul 17 05:31:37.009791 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:37.009727    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d09629f-8399-4354-ae12-785ee47c9b97" containerName="maas-api"
Jul 17 05:31:37.009791 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:37.009733    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5013f41a-a853-426d-a17e-0149d2ce7836" containerName="revoke-keys"
Jul 17 05:31:37.009791 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:37.009739    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5013f41a-a853-426d-a17e-0149d2ce7836" containerName="revoke-keys"
Jul 17 05:31:37.009791 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:37.009760    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15862f31-2950-4f3d-bf65-2473c0a25f96" containerName="revoke-keys"
Jul 17 05:31:37.009791 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:37.009764    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="15862f31-2950-4f3d-bf65-2473c0a25f96" containerName="revoke-keys"
Jul 17 05:31:37.009791 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:37.009776    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5013f41a-a853-426d-a17e-0149d2ce7836" containerName="revoke-keys"
Jul 17 05:31:37.009791 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:37.009781    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5013f41a-a853-426d-a17e-0149d2ce7836" containerName="revoke-keys"
Jul 17 05:31:37.010055 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:37.009834    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5013f41a-a853-426d-a17e-0149d2ce7836" containerName="revoke-keys"
Jul 17 05:31:37.010055 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:37.009842    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="15862f31-2950-4f3d-bf65-2473c0a25f96" containerName="revoke-keys"
Jul 17 05:31:37.010055 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:37.009850    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="15862f31-2950-4f3d-bf65-2473c0a25f96" containerName="revoke-keys"
Jul 17 05:31:37.010055 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:37.009857    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="15862f31-2950-4f3d-bf65-2473c0a25f96" containerName="revoke-keys"
Jul 17 05:31:37.010055 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:37.009863    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5013f41a-a853-426d-a17e-0149d2ce7836" containerName="revoke-keys"
Jul 17 05:31:37.014342 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:37.014323    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-tlwwq"
Jul 17 05:31:37.017114 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:37.017089    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qh8zj\""
Jul 17 05:31:37.020770 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:37.020746    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-tlwwq"]
Jul 17 05:31:37.121674 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:37.121640    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5da0886-0841-4ed2-93b3-41bf7535f560-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9eb8486e-tlwwq\" (UID: \"a5da0886-0841-4ed2-93b3-41bf7535f560\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-tlwwq"
Jul 17 05:31:37.222508 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:37.222466    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5da0886-0841-4ed2-93b3-41bf7535f560-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9eb8486e-tlwwq\" (UID: \"a5da0886-0841-4ed2-93b3-41bf7535f560\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-tlwwq"
Jul 17 05:31:37.223158 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:37.223132    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5da0886-0841-4ed2-93b3-41bf7535f560-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9eb8486e-tlwwq\" (UID: \"a5da0886-0841-4ed2-93b3-41bf7535f560\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-tlwwq"
Jul 17 05:31:37.326246 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:37.326210    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-tlwwq"
Jul 17 05:31:37.480326 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:37.480297    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-tlwwq"]
Jul 17 05:31:37.483407 ip-10-0-132-19 kubenswrapper[2562]: W0717 05:31:37.483381    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5da0886_0841_4ed2_93b3_41bf7535f560.slice/crio-90621da1fb544384d5832dc4292ce50ded03d3aa6ccf1f5176ffac9db18c04ed WatchSource:0}: Error finding container 90621da1fb544384d5832dc4292ce50ded03d3aa6ccf1f5176ffac9db18c04ed: Status 404 returned error can't find the container with id 90621da1fb544384d5832dc4292ce50ded03d3aa6ccf1f5176ffac9db18c04ed
Jul 17 05:31:38.240811 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:38.240776    2562 generic.go:356] "Generic (PLEG): container finished" podID="a5da0886-0841-4ed2-93b3-41bf7535f560" containerID="dd0cf4347049616e1c73f47cde71c7243793bb32b4920b63a194197c295d3627" exitCode=6
Jul 17 05:31:38.241234 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:38.240863    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-tlwwq" event={"ID":"a5da0886-0841-4ed2-93b3-41bf7535f560","Type":"ContainerDied","Data":"dd0cf4347049616e1c73f47cde71c7243793bb32b4920b63a194197c295d3627"}
Jul 17 05:31:38.241234 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:38.240924    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-tlwwq" event={"ID":"a5da0886-0841-4ed2-93b3-41bf7535f560","Type":"ContainerStarted","Data":"90621da1fb544384d5832dc4292ce50ded03d3aa6ccf1f5176ffac9db18c04ed"}
Jul 17 05:31:38.241234 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:38.241113    2562 scope.go:117] "RemoveContainer" containerID="dd0cf4347049616e1c73f47cde71c7243793bb32b4920b63a194197c295d3627"
Jul 17 05:31:39.247424 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:39.247382    2562 generic.go:356] "Generic (PLEG): container finished" podID="a5da0886-0841-4ed2-93b3-41bf7535f560" containerID="4cfe334d079681609c8364d0bb6318013cd8e0e45dcba71bad485d7f2ba8db83" exitCode=6
Jul 17 05:31:39.247856 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:39.247443    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-tlwwq" event={"ID":"a5da0886-0841-4ed2-93b3-41bf7535f560","Type":"ContainerDied","Data":"4cfe334d079681609c8364d0bb6318013cd8e0e45dcba71bad485d7f2ba8db83"}
Jul 17 05:31:39.247856 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:39.247503    2562 scope.go:117] "RemoveContainer" containerID="dd0cf4347049616e1c73f47cde71c7243793bb32b4920b63a194197c295d3627"
Jul 17 05:31:39.247856 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:39.247671    2562 scope.go:117] "RemoveContainer" containerID="4cfe334d079681609c8364d0bb6318013cd8e0e45dcba71bad485d7f2ba8db83"
Jul 17 05:31:39.247991 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:31:39.247906    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-9eb8486e-tlwwq_odh-ai-gateway-infra(a5da0886-0841-4ed2-93b3-41bf7535f560)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-tlwwq" podUID="a5da0886-0841-4ed2-93b3-41bf7535f560"
Jul 17 05:31:40.254421 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:40.254392    2562 scope.go:117] "RemoveContainer" containerID="4cfe334d079681609c8364d0bb6318013cd8e0e45dcba71bad485d7f2ba8db83"
Jul 17 05:31:40.254798 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:31:40.254599    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-9eb8486e-tlwwq_odh-ai-gateway-infra(a5da0886-0841-4ed2-93b3-41bf7535f560)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-tlwwq" podUID="a5da0886-0841-4ed2-93b3-41bf7535f560"
Jul 17 05:31:55.878931 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:55.878896    2562 scope.go:117] "RemoveContainer" containerID="4cfe334d079681609c8364d0bb6318013cd8e0e45dcba71bad485d7f2ba8db83"
Jul 17 05:31:56.340762 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:56.340727    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-tlwwq" event={"ID":"a5da0886-0841-4ed2-93b3-41bf7535f560","Type":"ContainerStarted","Data":"0431b291303f87bf11baef8444ba9f7cc71b4fd22e0a36039c01775d33b8519f"}
Jul 17 05:31:56.357343 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:56.357296    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-tlwwq" podStartSLOduration=20.357264006 podStartE2EDuration="20.357264006s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:31:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:31:56.357101334 +0000 UTC m=+2504.040695028" watchObservedRunningTime="2026-07-17 05:31:56.357264006 +0000 UTC m=+2504.040857669"
Jul 17 05:31:57.347242 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:57.347207    2562 generic.go:356] "Generic (PLEG): container finished" podID="a5da0886-0841-4ed2-93b3-41bf7535f560" containerID="0431b291303f87bf11baef8444ba9f7cc71b4fd22e0a36039c01775d33b8519f" exitCode=6
Jul 17 05:31:57.347626 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:57.347280    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-tlwwq" event={"ID":"a5da0886-0841-4ed2-93b3-41bf7535f560","Type":"ContainerDied","Data":"0431b291303f87bf11baef8444ba9f7cc71b4fd22e0a36039c01775d33b8519f"}
Jul 17 05:31:57.347626 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:57.347342    2562 scope.go:117] "RemoveContainer" containerID="4cfe334d079681609c8364d0bb6318013cd8e0e45dcba71bad485d7f2ba8db83"
Jul 17 05:31:57.347626 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:57.347623    2562 scope.go:117] "RemoveContainer" containerID="0431b291303f87bf11baef8444ba9f7cc71b4fd22e0a36039c01775d33b8519f"
Jul 17 05:31:57.347946 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:31:57.347917    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-9eb8486e-tlwwq_odh-ai-gateway-infra(a5da0886-0841-4ed2-93b3-41bf7535f560)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-tlwwq" podUID="a5da0886-0841-4ed2-93b3-41bf7535f560"
Jul 17 05:31:57.370581 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:57.370551    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-tlwwq"]
Jul 17 05:31:58.496111 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:58.496084    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-tlwwq"
Jul 17 05:31:58.621877 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:58.621775    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5da0886-0841-4ed2-93b3-41bf7535f560-maas-api-service-ca\") pod \"a5da0886-0841-4ed2-93b3-41bf7535f560\" (UID: \"a5da0886-0841-4ed2-93b3-41bf7535f560\") "
Jul 17 05:31:58.622173 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:58.622151    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5da0886-0841-4ed2-93b3-41bf7535f560-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a5da0886-0841-4ed2-93b3-41bf7535f560" (UID: "a5da0886-0841-4ed2-93b3-41bf7535f560"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:31:58.722518 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:58.722482    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5da0886-0841-4ed2-93b3-41bf7535f560-maas-api-service-ca\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:31:58.956340 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:58.956306    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-qz6fp"]
Jul 17 05:31:58.956719 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:58.956705    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5da0886-0841-4ed2-93b3-41bf7535f560" containerName="revoke-keys"
Jul 17 05:31:58.956768 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:58.956722    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5da0886-0841-4ed2-93b3-41bf7535f560" containerName="revoke-keys"
Jul 17 05:31:58.956768 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:58.956748    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15862f31-2950-4f3d-bf65-2473c0a25f96" containerName="revoke-keys"
Jul 17 05:31:58.956768 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:58.956754    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="15862f31-2950-4f3d-bf65-2473c0a25f96" containerName="revoke-keys"
Jul 17 05:31:58.956768 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:58.956761    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5013f41a-a853-426d-a17e-0149d2ce7836" containerName="revoke-keys"
Jul 17 05:31:58.956768 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:58.956766    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5013f41a-a853-426d-a17e-0149d2ce7836" containerName="revoke-keys"
Jul 17 05:31:58.956923 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:58.956771    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5da0886-0841-4ed2-93b3-41bf7535f560" containerName="revoke-keys"
Jul 17 05:31:58.956923 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:58.956776    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5da0886-0841-4ed2-93b3-41bf7535f560" containerName="revoke-keys"
Jul 17 05:31:58.956923 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:58.956838    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5da0886-0841-4ed2-93b3-41bf7535f560" containerName="revoke-keys"
Jul 17 05:31:58.956923 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:58.956846    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5013f41a-a853-426d-a17e-0149d2ce7836" containerName="revoke-keys"
Jul 17 05:31:58.956923 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:58.956853    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5da0886-0841-4ed2-93b3-41bf7535f560" containerName="revoke-keys"
Jul 17 05:31:58.956923 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:58.956862    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5da0886-0841-4ed2-93b3-41bf7535f560" containerName="revoke-keys"
Jul 17 05:31:58.960025 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:58.960008    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-qz6fp"
Jul 17 05:31:58.966630 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:58.966601    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-qz6fp"]
Jul 17 05:31:59.025011 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:59.024973    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4dccc764-df3a-4178-a124-c66444a8c7b8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-c8a383ce-qz6fp\" (UID: \"4dccc764-df3a-4178-a124-c66444a8c7b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-qz6fp"
Jul 17 05:31:59.126336 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:59.126290    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4dccc764-df3a-4178-a124-c66444a8c7b8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-c8a383ce-qz6fp\" (UID: \"4dccc764-df3a-4178-a124-c66444a8c7b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-qz6fp"
Jul 17 05:31:59.126901 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:59.126877    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4dccc764-df3a-4178-a124-c66444a8c7b8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-c8a383ce-qz6fp\" (UID: \"4dccc764-df3a-4178-a124-c66444a8c7b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-qz6fp"
Jul 17 05:31:59.272619 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:59.272524    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-qz6fp"
Jul 17 05:31:59.361802 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:59.361759    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-tlwwq" event={"ID":"a5da0886-0841-4ed2-93b3-41bf7535f560","Type":"ContainerDied","Data":"90621da1fb544384d5832dc4292ce50ded03d3aa6ccf1f5176ffac9db18c04ed"}
Jul 17 05:31:59.361952 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:59.361821    2562 scope.go:117] "RemoveContainer" containerID="0431b291303f87bf11baef8444ba9f7cc71b4fd22e0a36039c01775d33b8519f"
Jul 17 05:31:59.361952 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:59.361773    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-tlwwq"
Jul 17 05:31:59.389263 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:59.389234    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-tlwwq"]
Jul 17 05:31:59.394383 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:59.394357    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-tlwwq"]
Jul 17 05:31:59.410132 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:31:59.410108    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-qz6fp"]
Jul 17 05:31:59.412163 ip-10-0-132-19 kubenswrapper[2562]: W0717 05:31:59.412124    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dccc764_df3a_4178_a124_c66444a8c7b8.slice/crio-8f5d42f57890dc1c60b5f1f9a76d2d0cde9b95b056edad54f2db7f72c06ae0fc WatchSource:0}: Error finding container 8f5d42f57890dc1c60b5f1f9a76d2d0cde9b95b056edad54f2db7f72c06ae0fc: Status 404 returned error can't find the container with id 8f5d42f57890dc1c60b5f1f9a76d2d0cde9b95b056edad54f2db7f72c06ae0fc
Jul 17 05:32:00.369321 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:00.369284    2562 generic.go:356] "Generic (PLEG): container finished" podID="4dccc764-df3a-4178-a124-c66444a8c7b8" containerID="5f781e862b9d2ba874ca6b3548199aa01e72aa92c0cb4f8dff0c56abac3c88f5" exitCode=6
Jul 17 05:32:00.369722 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:00.369331    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-qz6fp" event={"ID":"4dccc764-df3a-4178-a124-c66444a8c7b8","Type":"ContainerDied","Data":"5f781e862b9d2ba874ca6b3548199aa01e72aa92c0cb4f8dff0c56abac3c88f5"}
Jul 17 05:32:00.369722 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:00.369367    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-qz6fp" event={"ID":"4dccc764-df3a-4178-a124-c66444a8c7b8","Type":"ContainerStarted","Data":"8f5d42f57890dc1c60b5f1f9a76d2d0cde9b95b056edad54f2db7f72c06ae0fc"}
Jul 17 05:32:00.369722 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:00.369552    2562 scope.go:117] "RemoveContainer" containerID="5f781e862b9d2ba874ca6b3548199aa01e72aa92c0cb4f8dff0c56abac3c88f5"
Jul 17 05:32:00.884236 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:00.884199    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5da0886-0841-4ed2-93b3-41bf7535f560" path="/var/lib/kubelet/pods/a5da0886-0841-4ed2-93b3-41bf7535f560/volumes"
Jul 17 05:32:01.376355 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:01.376314    2562 generic.go:356] "Generic (PLEG): container finished" podID="4dccc764-df3a-4178-a124-c66444a8c7b8" containerID="fc4d9309254eb6a50f56e5a656c5c1073a1d0ebc478e4d82ad36818ae03445f4" exitCode=6
Jul 17 05:32:01.376860 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:01.376361    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-qz6fp" event={"ID":"4dccc764-df3a-4178-a124-c66444a8c7b8","Type":"ContainerDied","Data":"fc4d9309254eb6a50f56e5a656c5c1073a1d0ebc478e4d82ad36818ae03445f4"}
Jul 17 05:32:01.376860 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:01.376418    2562 scope.go:117] "RemoveContainer" containerID="5f781e862b9d2ba874ca6b3548199aa01e72aa92c0cb4f8dff0c56abac3c88f5"
Jul 17 05:32:01.376860 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:01.376623    2562 scope.go:117] "RemoveContainer" containerID="fc4d9309254eb6a50f56e5a656c5c1073a1d0ebc478e4d82ad36818ae03445f4"
Jul 17 05:32:01.376860 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:32:01.376848    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-c8a383ce-qz6fp_odh-ai-gateway-infra(4dccc764-df3a-4178-a124-c66444a8c7b8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-qz6fp" podUID="4dccc764-df3a-4178-a124-c66444a8c7b8"
Jul 17 05:32:02.382589 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:02.382563    2562 scope.go:117] "RemoveContainer" containerID="fc4d9309254eb6a50f56e5a656c5c1073a1d0ebc478e4d82ad36818ae03445f4"
Jul 17 05:32:02.383021 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:32:02.382876    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-c8a383ce-qz6fp_odh-ai-gateway-infra(4dccc764-df3a-4178-a124-c66444a8c7b8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-qz6fp" podUID="4dccc764-df3a-4178-a124-c66444a8c7b8"
Jul 17 05:32:04.421574 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:04.421490    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-khh6l"]
Jul 17 05:32:04.422061 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:04.421898    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5da0886-0841-4ed2-93b3-41bf7535f560" containerName="revoke-keys"
Jul 17 05:32:04.422061 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:04.421908    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5da0886-0841-4ed2-93b3-41bf7535f560" containerName="revoke-keys"
Jul 17 05:32:04.425911 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:04.425895    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-khh6l"
Jul 17 05:32:04.455297 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:04.455267    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-khh6l"]
Jul 17 05:32:04.580461 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:04.580420    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7df963d-37d1-4f83-afc9-053e37e0c5eb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-bf0ad9db-khh6l\" (UID: \"a7df963d-37d1-4f83-afc9-053e37e0c5eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-khh6l"
Jul 17 05:32:04.681150 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:04.681056    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7df963d-37d1-4f83-afc9-053e37e0c5eb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-bf0ad9db-khh6l\" (UID: \"a7df963d-37d1-4f83-afc9-053e37e0c5eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-khh6l"
Jul 17 05:32:04.681638 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:04.681618    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7df963d-37d1-4f83-afc9-053e37e0c5eb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-bf0ad9db-khh6l\" (UID: \"a7df963d-37d1-4f83-afc9-053e37e0c5eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-khh6l"
Jul 17 05:32:04.735537 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:04.735493    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-khh6l"
Jul 17 05:32:04.873935 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:04.873906    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-khh6l"]
Jul 17 05:32:04.875702 ip-10-0-132-19 kubenswrapper[2562]: W0717 05:32:04.875672    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7df963d_37d1_4f83_afc9_053e37e0c5eb.slice/crio-62a170103a84bc2149dcee556700edca4280ccc15b0ddd3caa105cc1d77b588b WatchSource:0}: Error finding container 62a170103a84bc2149dcee556700edca4280ccc15b0ddd3caa105cc1d77b588b: Status 404 returned error can't find the container with id 62a170103a84bc2149dcee556700edca4280ccc15b0ddd3caa105cc1d77b588b
Jul 17 05:32:05.399130 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:05.399094    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-khh6l" event={"ID":"a7df963d-37d1-4f83-afc9-053e37e0c5eb","Type":"ContainerStarted","Data":"d6e3c873f582c64273549d8cac789301b1049cb5ff9e550eb1f211c6545fbbd6"}
Jul 17 05:32:05.399317 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:05.399147    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-khh6l" event={"ID":"a7df963d-37d1-4f83-afc9-053e37e0c5eb","Type":"ContainerStarted","Data":"62a170103a84bc2149dcee556700edca4280ccc15b0ddd3caa105cc1d77b588b"}
Jul 17 05:32:05.418325 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:05.418270    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-khh6l" podStartSLOduration=1.418254116 podStartE2EDuration="1.418254116s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:32:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:32:05.416900986 +0000 UTC m=+2513.100494644" watchObservedRunningTime="2026-07-17 05:32:05.418254116 +0000 UTC m=+2513.101847779"
Jul 17 05:32:06.405665 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:06.405623    2562 generic.go:356] "Generic (PLEG): container finished" podID="a7df963d-37d1-4f83-afc9-053e37e0c5eb" containerID="d6e3c873f582c64273549d8cac789301b1049cb5ff9e550eb1f211c6545fbbd6" exitCode=6
Jul 17 05:32:06.406154 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:06.405709    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-khh6l" event={"ID":"a7df963d-37d1-4f83-afc9-053e37e0c5eb","Type":"ContainerDied","Data":"d6e3c873f582c64273549d8cac789301b1049cb5ff9e550eb1f211c6545fbbd6"}
Jul 17 05:32:06.406154 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:06.405864    2562 scope.go:117] "RemoveContainer" containerID="d6e3c873f582c64273549d8cac789301b1049cb5ff9e550eb1f211c6545fbbd6"
Jul 17 05:32:07.413209 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:07.413168    2562 generic.go:356] "Generic (PLEG): container finished" podID="a7df963d-37d1-4f83-afc9-053e37e0c5eb" containerID="c8b882c38891593f3aa7bbd7689813625ab1a2de3892a8644fe13c1a96b4aba9" exitCode=6
Jul 17 05:32:07.413209 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:07.413211    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-khh6l" event={"ID":"a7df963d-37d1-4f83-afc9-053e37e0c5eb","Type":"ContainerDied","Data":"c8b882c38891593f3aa7bbd7689813625ab1a2de3892a8644fe13c1a96b4aba9"}
Jul 17 05:32:07.413663 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:07.413263    2562 scope.go:117] "RemoveContainer" containerID="d6e3c873f582c64273549d8cac789301b1049cb5ff9e550eb1f211c6545fbbd6"
Jul 17 05:32:07.413663 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:07.413485    2562 scope.go:117] "RemoveContainer" containerID="c8b882c38891593f3aa7bbd7689813625ab1a2de3892a8644fe13c1a96b4aba9"
Jul 17 05:32:07.413744 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:32:07.413707    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-bf0ad9db-khh6l_odh-ai-gateway-infra(a7df963d-37d1-4f83-afc9-053e37e0c5eb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-khh6l" podUID="a7df963d-37d1-4f83-afc9-053e37e0c5eb"
Jul 17 05:32:08.419921 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:08.419891    2562 scope.go:117] "RemoveContainer" containerID="c8b882c38891593f3aa7bbd7689813625ab1a2de3892a8644fe13c1a96b4aba9"
Jul 17 05:32:08.420306 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:32:08.420095    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-bf0ad9db-khh6l_odh-ai-gateway-infra(a7df963d-37d1-4f83-afc9-053e37e0c5eb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-khh6l" podUID="a7df963d-37d1-4f83-afc9-053e37e0c5eb"
Jul 17 05:32:10.834661 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:10.834616    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-4nj99"]
Jul 17 05:32:10.839884 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:10.839864    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-4nj99"
Jul 17 05:32:10.846825 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:10.846798    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-4nj99"]
Jul 17 05:32:10.941476 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:10.941440    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c29f22f-0d63-45e3-ba82-2e851ccbcbae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4e221674-4nj99\" (UID: \"5c29f22f-0d63-45e3-ba82-2e851ccbcbae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-4nj99"
Jul 17 05:32:11.042791 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:11.042750    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c29f22f-0d63-45e3-ba82-2e851ccbcbae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4e221674-4nj99\" (UID: \"5c29f22f-0d63-45e3-ba82-2e851ccbcbae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-4nj99"
Jul 17 05:32:11.043459 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:11.043438    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c29f22f-0d63-45e3-ba82-2e851ccbcbae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4e221674-4nj99\" (UID: \"5c29f22f-0d63-45e3-ba82-2e851ccbcbae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-4nj99"
Jul 17 05:32:11.152174 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:11.152084    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-4nj99"
Jul 17 05:32:11.284664 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:11.284632    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-4nj99"]
Jul 17 05:32:11.285988 ip-10-0-132-19 kubenswrapper[2562]: W0717 05:32:11.285934    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c29f22f_0d63_45e3_ba82_2e851ccbcbae.slice/crio-202a005c811841a6429efd032ba100612e13e7ed31cb59602fdafa4b26639266 WatchSource:0}: Error finding container 202a005c811841a6429efd032ba100612e13e7ed31cb59602fdafa4b26639266: Status 404 returned error can't find the container with id 202a005c811841a6429efd032ba100612e13e7ed31cb59602fdafa4b26639266
Jul 17 05:32:11.435625 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:11.435533    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-4nj99" event={"ID":"5c29f22f-0d63-45e3-ba82-2e851ccbcbae","Type":"ContainerStarted","Data":"202a005c811841a6429efd032ba100612e13e7ed31cb59602fdafa4b26639266"}
Jul 17 05:32:12.442185 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:12.442154    2562 generic.go:356] "Generic (PLEG): container finished" podID="5c29f22f-0d63-45e3-ba82-2e851ccbcbae" containerID="a50b5e917d0b4a361fd86b6805de792e0ce2716b29fb2c1562783c0e4f055ce7" exitCode=6
Jul 17 05:32:12.442575 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:12.442250    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-4nj99" event={"ID":"5c29f22f-0d63-45e3-ba82-2e851ccbcbae","Type":"ContainerDied","Data":"a50b5e917d0b4a361fd86b6805de792e0ce2716b29fb2c1562783c0e4f055ce7"}
Jul 17 05:32:12.442575 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:12.442400    2562 scope.go:117] "RemoveContainer" containerID="a50b5e917d0b4a361fd86b6805de792e0ce2716b29fb2c1562783c0e4f055ce7"
Jul 17 05:32:12.880984 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:12.880932    2562 scope.go:117] "RemoveContainer" containerID="fc4d9309254eb6a50f56e5a656c5c1073a1d0ebc478e4d82ad36818ae03445f4"
Jul 17 05:32:13.449727 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:13.449690    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-qz6fp" event={"ID":"4dccc764-df3a-4178-a124-c66444a8c7b8","Type":"ContainerStarted","Data":"f4ffdffd7f2f92d529bb45b18b8f012cd97a20082f27e03a63c658dee6899c30"}
Jul 17 05:32:13.451504 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:13.451479    2562 generic.go:356] "Generic (PLEG): container finished" podID="5c29f22f-0d63-45e3-ba82-2e851ccbcbae" containerID="200917ea2f2cbf0f39774cc341bdb79b5aece0292a0784b406a97e2c91563e17" exitCode=6
Jul 17 05:32:13.451660 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:13.451564    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-4nj99" event={"ID":"5c29f22f-0d63-45e3-ba82-2e851ccbcbae","Type":"ContainerDied","Data":"200917ea2f2cbf0f39774cc341bdb79b5aece0292a0784b406a97e2c91563e17"}
Jul 17 05:32:13.451660 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:13.451618    2562 scope.go:117] "RemoveContainer" containerID="a50b5e917d0b4a361fd86b6805de792e0ce2716b29fb2c1562783c0e4f055ce7"
Jul 17 05:32:13.451768 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:13.451723    2562 scope.go:117] "RemoveContainer" containerID="200917ea2f2cbf0f39774cc341bdb79b5aece0292a0784b406a97e2c91563e17"
Jul 17 05:32:13.451915 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:32:13.451896    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-4e221674-4nj99_odh-ai-gateway-infra(5c29f22f-0d63-45e3-ba82-2e851ccbcbae)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-4nj99" podUID="5c29f22f-0d63-45e3-ba82-2e851ccbcbae"
Jul 17 05:32:13.466952 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:13.466910    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-qz6fp" podStartSLOduration=15.466896789 podStartE2EDuration="15.466896789s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:31:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:32:13.46601816 +0000 UTC m=+2521.149611839" watchObservedRunningTime="2026-07-17 05:32:13.466896789 +0000 UTC m=+2521.150490446"
Jul 17 05:32:14.458559 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:14.458521    2562 generic.go:356] "Generic (PLEG): container finished" podID="4dccc764-df3a-4178-a124-c66444a8c7b8" containerID="f4ffdffd7f2f92d529bb45b18b8f012cd97a20082f27e03a63c658dee6899c30" exitCode=6
Jul 17 05:32:14.459076 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:14.458598    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-qz6fp" event={"ID":"4dccc764-df3a-4178-a124-c66444a8c7b8","Type":"ContainerDied","Data":"f4ffdffd7f2f92d529bb45b18b8f012cd97a20082f27e03a63c658dee6899c30"}
Jul 17 05:32:14.459076 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:14.458706    2562 scope.go:117] "RemoveContainer" containerID="fc4d9309254eb6a50f56e5a656c5c1073a1d0ebc478e4d82ad36818ae03445f4"
Jul 17 05:32:14.459076 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:14.458978    2562 scope.go:117] "RemoveContainer" containerID="f4ffdffd7f2f92d529bb45b18b8f012cd97a20082f27e03a63c658dee6899c30"
Jul 17 05:32:14.459268 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:32:14.459246    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-c8a383ce-qz6fp_odh-ai-gateway-infra(4dccc764-df3a-4178-a124-c66444a8c7b8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-qz6fp" podUID="4dccc764-df3a-4178-a124-c66444a8c7b8"
Jul 17 05:32:14.460365 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:14.460350    2562 scope.go:117] "RemoveContainer" containerID="200917ea2f2cbf0f39774cc341bdb79b5aece0292a0784b406a97e2c91563e17"
Jul 17 05:32:14.460559 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:32:14.460541    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-4e221674-4nj99_odh-ai-gateway-infra(5c29f22f-0d63-45e3-ba82-2e851ccbcbae)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-4nj99" podUID="5c29f22f-0d63-45e3-ba82-2e851ccbcbae"
Jul 17 05:32:14.477083 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:14.477035    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-qz6fp"]
Jul 17 05:32:15.603058 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:15.603035    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-qz6fp"
Jul 17 05:32:15.682489 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:15.682455    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4dccc764-df3a-4178-a124-c66444a8c7b8-maas-api-service-ca\") pod \"4dccc764-df3a-4178-a124-c66444a8c7b8\" (UID: \"4dccc764-df3a-4178-a124-c66444a8c7b8\") "
Jul 17 05:32:15.682867 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:15.682846    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4dccc764-df3a-4178-a124-c66444a8c7b8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4dccc764-df3a-4178-a124-c66444a8c7b8" (UID: "4dccc764-df3a-4178-a124-c66444a8c7b8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:32:15.783937 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:15.783851    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4dccc764-df3a-4178-a124-c66444a8c7b8-maas-api-service-ca\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:32:16.473683 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:16.473649    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-qz6fp" event={"ID":"4dccc764-df3a-4178-a124-c66444a8c7b8","Type":"ContainerDied","Data":"8f5d42f57890dc1c60b5f1f9a76d2d0cde9b95b056edad54f2db7f72c06ae0fc"}
Jul 17 05:32:16.473865 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:16.473706    2562 scope.go:117] "RemoveContainer" containerID="f4ffdffd7f2f92d529bb45b18b8f012cd97a20082f27e03a63c658dee6899c30"
Jul 17 05:32:16.473865 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:16.473665    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-qz6fp"
Jul 17 05:32:16.502118 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:16.502078    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-qz6fp"]
Jul 17 05:32:16.506223 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:16.506198    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-qz6fp"]
Jul 17 05:32:16.573944 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:16.573913    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5"]
Jul 17 05:32:16.574380 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:16.574365    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4dccc764-df3a-4178-a124-c66444a8c7b8" containerName="revoke-keys"
Jul 17 05:32:16.574448 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:16.574386    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4dccc764-df3a-4178-a124-c66444a8c7b8" containerName="revoke-keys"
Jul 17 05:32:16.574448 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:16.574411    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4dccc764-df3a-4178-a124-c66444a8c7b8" containerName="revoke-keys"
Jul 17 05:32:16.574448 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:16.574416    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4dccc764-df3a-4178-a124-c66444a8c7b8" containerName="revoke-keys"
Jul 17 05:32:16.574539 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:16.574507    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4dccc764-df3a-4178-a124-c66444a8c7b8" containerName="revoke-keys"
Jul 17 05:32:16.574539 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:16.574517    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4dccc764-df3a-4178-a124-c66444a8c7b8" containerName="revoke-keys"
Jul 17 05:32:16.579280 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:16.579258    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5"
Jul 17 05:32:16.582234 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:16.582215    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-857d512b-openshift-default-dockercfg-bmb7f\""
Jul 17 05:32:16.590434 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:16.590411    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-istio-envoy\") pod \"e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5\" (UID: \"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d\") " pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5"
Jul 17 05:32:16.590538 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:16.590443    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-workload-certs\") pod \"e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5\" (UID: \"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d\") " pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5"
Jul 17 05:32:16.590580 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:16.590541    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-istio-podinfo\") pod \"e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5\" (UID: \"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d\") " pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5"
Jul 17 05:32:16.590617 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:16.590577    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-istio-data\") pod \"e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5\" (UID: \"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d\") " pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5"
Jul 17 05:32:16.590653 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:16.590638    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-workload-socket\") pod \"e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5\" (UID: \"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d\") " pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5"
Jul 17 05:32:16.590685 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:16.590659    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-credential-socket\") pod \"e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5\" (UID: \"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d\") " pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5"
Jul 17 05:32:16.590721 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:16.590710    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-istiod-ca-cert\") pod \"e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5\" (UID: \"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d\") " pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5"
Jul 17 05:32:16.590756 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:16.590738    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhhjl\" (UniqueName: \"kubernetes.io/projected/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-kube-api-access-jhhjl\") pod \"e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5\" (UID: \"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d\") " pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5"
Jul 17 05:32:16.590790 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:16.590764    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-istio-token\") pod \"e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5\" (UID: \"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d\") " pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5"
Jul 17 05:32:16.591049 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:16.591030    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5"]
Jul 17 05:32:16.691482 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:16.691451    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-istio-podinfo\") pod \"e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5\" (UID: \"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d\") " pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5"
Jul 17 05:32:16.691923 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:16.691487    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-istio-data\") pod \"e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5\" (UID: \"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d\") " pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5"
Jul 17 05:32:16.691923 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:16.691519    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-workload-socket\") pod \"e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5\" (UID: \"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d\") " pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5"
Jul 17 05:32:16.691923 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:16.691535    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-credential-socket\") pod \"e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5\" (UID: \"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d\") " pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5"
Jul 17 05:32:16.691923 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:16.691566    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-istiod-ca-cert\") pod \"e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5\" (UID: \"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d\") " pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5"
Jul 17 05:32:16.691923 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:16.691601    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhhjl\" (UniqueName: \"kubernetes.io/projected/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-kube-api-access-jhhjl\") pod \"e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5\" (UID: \"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d\") " pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5"
Jul 17 05:32:16.691923 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:16.691618    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-istio-token\") pod \"e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5\" (UID: \"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d\") " pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5"
Jul 17 05:32:16.691923 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:16.691661    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-istio-envoy\") pod \"e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5\" (UID: \"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d\") " pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5"
Jul 17 05:32:16.691923 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:16.691688    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-workload-certs\") pod \"e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5\" (UID: \"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d\") " pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5"
Jul 17 05:32:16.692327 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:16.691989    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-credential-socket\") pod \"e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5\" (UID: \"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d\") " pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5"
Jul 17 05:32:16.692327 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:16.692074    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-workload-certs\") pod \"e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5\" (UID: \"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d\") " pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5"
Jul 17 05:32:16.692327 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:16.692097    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-workload-socket\") pod \"e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5\" (UID: \"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d\") " pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5"
Jul 17 05:32:16.692327 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:16.692117    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-istio-data\") pod \"e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5\" (UID: \"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d\") " pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5"
Jul 17 05:32:16.692458 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:16.692365    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-istiod-ca-cert\") pod \"e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5\" (UID: \"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d\") " pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5"
Jul 17 05:32:16.694045 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:16.694024    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-istio-envoy\") pod \"e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5\" (UID: \"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d\") " pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5"
Jul 17 05:32:16.694150 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:16.694120    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-istio-podinfo\") pod \"e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5\" (UID: \"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d\") " pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5"
Jul 17 05:32:16.700071 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:16.700049    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhhjl\" (UniqueName: \"kubernetes.io/projected/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-kube-api-access-jhhjl\") pod \"e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5\" (UID: \"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d\") " pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5"
Jul 17 05:32:16.700071 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:16.700059    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-istio-token\") pod \"e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5\" (UID: \"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d\") " pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5"
Jul 17 05:32:16.884545 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:16.884509    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4dccc764-df3a-4178-a124-c66444a8c7b8" path="/var/lib/kubelet/pods/4dccc764-df3a-4178-a124-c66444a8c7b8/volumes"
Jul 17 05:32:16.890486 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:16.890462    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5"
Jul 17 05:32:17.033936 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:17.033906    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5"]
Jul 17 05:32:17.035617 ip-10-0-132-19 kubenswrapper[2562]: W0717 05:32:17.035594    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode875c9d9_5339_4913_9a05_2bc9e6a0fe4d.slice/crio-c5d8883f316e95f840cd76016df0841cda9ccfdd2f27bf05d72edb30664811b3 WatchSource:0}: Error finding container c5d8883f316e95f840cd76016df0841cda9ccfdd2f27bf05d72edb30664811b3: Status 404 returned error can't find the container with id c5d8883f316e95f840cd76016df0841cda9ccfdd2f27bf05d72edb30664811b3
Jul 17 05:32:17.038317 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:17.038286    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:32:17.038635 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:17.038369    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:32:17.038635 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:17.038438    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:32:17.481226 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:17.481182    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5" event={"ID":"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d","Type":"ContainerStarted","Data":"a5e4c1bdd8557a5da5d62275218add43a4fb58fe1e0b57ed0d9b6d348ab6eb84"}
Jul 17 05:32:17.481384 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:17.481249    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5" event={"ID":"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d","Type":"ContainerStarted","Data":"c5d8883f316e95f840cd76016df0841cda9ccfdd2f27bf05d72edb30664811b3"}
Jul 17 05:32:17.503908 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:17.503848    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5" podStartSLOduration=1.503828739 podStartE2EDuration="1.503828739s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:32:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:32:17.502541607 +0000 UTC m=+2525.186135293" watchObservedRunningTime="2026-07-17 05:32:17.503828739 +0000 UTC m=+2525.187422397"
Jul 17 05:32:17.891064 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:17.891031    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5"
Jul 17 05:32:18.896504 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:18.896475    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5"
Jul 17 05:32:19.491692 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:19.491659    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5"
Jul 17 05:32:19.492584 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:19.492566    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5"
Jul 17 05:32:19.710648 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:19.710605    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-857d512b-7567bf77d9-6qq2b"]
Jul 17 05:32:19.711089 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:19.711074    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4dccc764-df3a-4178-a124-c66444a8c7b8" containerName="revoke-keys"
Jul 17 05:32:19.711089 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:19.711089    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4dccc764-df3a-4178-a124-c66444a8c7b8" containerName="revoke-keys"
Jul 17 05:32:19.711222 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:19.711155    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4dccc764-df3a-4178-a124-c66444a8c7b8" containerName="revoke-keys"
Jul 17 05:32:19.714301 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:19.714282    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-857d512b-7567bf77d9-6qq2b"
Jul 17 05:32:19.716746 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:19.716725    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-857d512b-serving-cert\""
Jul 17 05:32:19.716746 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:19.716740    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-xwcd7\""
Jul 17 05:32:19.725231 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:19.725205    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-857d512b-7567bf77d9-6qq2b"]
Jul 17 05:32:19.824535 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:19.824497    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40c39234-3495-4621-ac2a-76503def4b04-openshift-service-ca\") pod \"maas-api-e2e-mt-857d512b-7567bf77d9-6qq2b\" (UID: \"40c39234-3495-4621-ac2a-76503def4b04\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-857d512b-7567bf77d9-6qq2b"
Jul 17 05:32:19.824742 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:19.824658    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40c39234-3495-4621-ac2a-76503def4b04-maas-api-tls\") pod \"maas-api-e2e-mt-857d512b-7567bf77d9-6qq2b\" (UID: \"40c39234-3495-4621-ac2a-76503def4b04\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-857d512b-7567bf77d9-6qq2b"
Jul 17 05:32:19.824813 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:19.824774    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgjs5\" (UniqueName: \"kubernetes.io/projected/40c39234-3495-4621-ac2a-76503def4b04-kube-api-access-xgjs5\") pod \"maas-api-e2e-mt-857d512b-7567bf77d9-6qq2b\" (UID: \"40c39234-3495-4621-ac2a-76503def4b04\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-857d512b-7567bf77d9-6qq2b"
Jul 17 05:32:19.878663 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:19.878627    2562 scope.go:117] "RemoveContainer" containerID="c8b882c38891593f3aa7bbd7689813625ab1a2de3892a8644fe13c1a96b4aba9"
Jul 17 05:32:19.926101 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:19.926071    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40c39234-3495-4621-ac2a-76503def4b04-openshift-service-ca\") pod \"maas-api-e2e-mt-857d512b-7567bf77d9-6qq2b\" (UID: \"40c39234-3495-4621-ac2a-76503def4b04\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-857d512b-7567bf77d9-6qq2b"
Jul 17 05:32:19.926484 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:19.926155    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40c39234-3495-4621-ac2a-76503def4b04-maas-api-tls\") pod \"maas-api-e2e-mt-857d512b-7567bf77d9-6qq2b\" (UID: \"40c39234-3495-4621-ac2a-76503def4b04\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-857d512b-7567bf77d9-6qq2b"
Jul 17 05:32:19.926484 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:19.926200    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgjs5\" (UniqueName: \"kubernetes.io/projected/40c39234-3495-4621-ac2a-76503def4b04-kube-api-access-xgjs5\") pod \"maas-api-e2e-mt-857d512b-7567bf77d9-6qq2b\" (UID: \"40c39234-3495-4621-ac2a-76503def4b04\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-857d512b-7567bf77d9-6qq2b"
Jul 17 05:32:19.926484 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:32:19.926327    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-857d512b-serving-cert: secret "maas-api-e2e-mt-857d512b-serving-cert" not found
Jul 17 05:32:19.926484 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:32:19.926416    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40c39234-3495-4621-ac2a-76503def4b04-maas-api-tls podName:40c39234-3495-4621-ac2a-76503def4b04 nodeName:}" failed. No retries permitted until 2026-07-17 05:32:20.42639361 +0000 UTC m=+2528.109987262 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/40c39234-3495-4621-ac2a-76503def4b04-maas-api-tls") pod "maas-api-e2e-mt-857d512b-7567bf77d9-6qq2b" (UID: "40c39234-3495-4621-ac2a-76503def4b04") : secret "maas-api-e2e-mt-857d512b-serving-cert" not found
Jul 17 05:32:19.926705 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:19.926647    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40c39234-3495-4621-ac2a-76503def4b04-openshift-service-ca\") pod \"maas-api-e2e-mt-857d512b-7567bf77d9-6qq2b\" (UID: \"40c39234-3495-4621-ac2a-76503def4b04\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-857d512b-7567bf77d9-6qq2b"
Jul 17 05:32:19.935576 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:19.935547    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgjs5\" (UniqueName: \"kubernetes.io/projected/40c39234-3495-4621-ac2a-76503def4b04-kube-api-access-xgjs5\") pod \"maas-api-e2e-mt-857d512b-7567bf77d9-6qq2b\" (UID: \"40c39234-3495-4621-ac2a-76503def4b04\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-857d512b-7567bf77d9-6qq2b"
Jul 17 05:32:20.431804 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:20.431715    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40c39234-3495-4621-ac2a-76503def4b04-maas-api-tls\") pod \"maas-api-e2e-mt-857d512b-7567bf77d9-6qq2b\" (UID: \"40c39234-3495-4621-ac2a-76503def4b04\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-857d512b-7567bf77d9-6qq2b"
Jul 17 05:32:20.434239 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:20.434218    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40c39234-3495-4621-ac2a-76503def4b04-maas-api-tls\") pod \"maas-api-e2e-mt-857d512b-7567bf77d9-6qq2b\" (UID: \"40c39234-3495-4621-ac2a-76503def4b04\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-857d512b-7567bf77d9-6qq2b"
Jul 17 05:32:20.498789 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:20.498747    2562 generic.go:356] "Generic (PLEG): container finished" podID="a7df963d-37d1-4f83-afc9-053e37e0c5eb" containerID="cdb5370e2a704356f5c68a90cd751e6238871b4da3b7a656200fbc8e43f8f1dc" exitCode=6
Jul 17 05:32:20.499034 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:20.498832    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-khh6l" event={"ID":"a7df963d-37d1-4f83-afc9-053e37e0c5eb","Type":"ContainerDied","Data":"cdb5370e2a704356f5c68a90cd751e6238871b4da3b7a656200fbc8e43f8f1dc"}
Jul 17 05:32:20.499034 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:20.498904    2562 scope.go:117] "RemoveContainer" containerID="c8b882c38891593f3aa7bbd7689813625ab1a2de3892a8644fe13c1a96b4aba9"
Jul 17 05:32:20.499349 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:20.499331    2562 scope.go:117] "RemoveContainer" containerID="cdb5370e2a704356f5c68a90cd751e6238871b4da3b7a656200fbc8e43f8f1dc"
Jul 17 05:32:20.499593 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:32:20.499574    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-bf0ad9db-khh6l_odh-ai-gateway-infra(a7df963d-37d1-4f83-afc9-053e37e0c5eb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-khh6l" podUID="a7df963d-37d1-4f83-afc9-053e37e0c5eb"
Jul 17 05:32:20.627017 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:20.626980    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-857d512b-7567bf77d9-6qq2b"
Jul 17 05:32:20.779859 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:20.779829    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-857d512b-7567bf77d9-6qq2b"]
Jul 17 05:32:20.782064 ip-10-0-132-19 kubenswrapper[2562]: W0717 05:32:20.782018    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40c39234_3495_4621_ac2a_76503def4b04.slice/crio-0f732dcd0e72b6f4d7e7c47724ac2b726eff5e211ceac57c73d10f71bcfc86f3 WatchSource:0}: Error finding container 0f732dcd0e72b6f4d7e7c47724ac2b726eff5e211ceac57c73d10f71bcfc86f3: Status 404 returned error can't find the container with id 0f732dcd0e72b6f4d7e7c47724ac2b726eff5e211ceac57c73d10f71bcfc86f3
Jul 17 05:32:20.783602 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:20.783578    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 05:32:21.509177 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:21.509079    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-857d512b-7567bf77d9-6qq2b" event={"ID":"40c39234-3495-4621-ac2a-76503def4b04","Type":"ContainerStarted","Data":"622b41c273f8f19fbb4412bcd3eb0da4575ac67200a78ba7c0ce5306b24a9dad"}
Jul 17 05:32:21.509177 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:21.509132    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-857d512b-7567bf77d9-6qq2b" event={"ID":"40c39234-3495-4621-ac2a-76503def4b04","Type":"ContainerStarted","Data":"0f732dcd0e72b6f4d7e7c47724ac2b726eff5e211ceac57c73d10f71bcfc86f3"}
Jul 17 05:32:21.509642 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:21.509184    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-857d512b-7567bf77d9-6qq2b"
Jul 17 05:32:21.537987 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:21.537942    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-khh6l"]
Jul 17 05:32:21.547850 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:21.547802    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-857d512b-7567bf77d9-6qq2b" podStartSLOduration=2.093661817 podStartE2EDuration="2.54778742s" totalImagesPullingTime="454.125603ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:32:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:32:21.547031256 +0000 UTC m=+2529.230625012" watchObservedRunningTime="2026-07-17 05:32:21.54778742 +0000 UTC m=+2529.231381109"
Jul 17 05:32:21.679288 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:21.679260    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-khh6l"
Jul 17 05:32:21.744391 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:21.744352    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7df963d-37d1-4f83-afc9-053e37e0c5eb-maas-api-service-ca\") pod \"a7df963d-37d1-4f83-afc9-053e37e0c5eb\" (UID: \"a7df963d-37d1-4f83-afc9-053e37e0c5eb\") "
Jul 17 05:32:21.744688 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:21.744667    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7df963d-37d1-4f83-afc9-053e37e0c5eb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a7df963d-37d1-4f83-afc9-053e37e0c5eb" (UID: "a7df963d-37d1-4f83-afc9-053e37e0c5eb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:32:21.845903 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:21.845873    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7df963d-37d1-4f83-afc9-053e37e0c5eb-maas-api-service-ca\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:32:22.517286 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:22.517253    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-khh6l"
Jul 17 05:32:22.517728 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:22.517290    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-khh6l" event={"ID":"a7df963d-37d1-4f83-afc9-053e37e0c5eb","Type":"ContainerDied","Data":"62a170103a84bc2149dcee556700edca4280ccc15b0ddd3caa105cc1d77b588b"}
Jul 17 05:32:22.517728 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:22.517348    2562 scope.go:117] "RemoveContainer" containerID="cdb5370e2a704356f5c68a90cd751e6238871b4da3b7a656200fbc8e43f8f1dc"
Jul 17 05:32:22.540118 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:22.540090    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-khh6l"]
Jul 17 05:32:22.545162 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:22.545137    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-khh6l"]
Jul 17 05:32:22.884639 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:22.884598    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7df963d-37d1-4f83-afc9-053e37e0c5eb" path="/var/lib/kubelet/pods/a7df963d-37d1-4f83-afc9-053e37e0c5eb/volumes"
Jul 17 05:32:27.523041 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:27.523008    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-857d512b-7567bf77d9-6qq2b"
Jul 17 05:32:28.878594 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:28.878563    2562 scope.go:117] "RemoveContainer" containerID="200917ea2f2cbf0f39774cc341bdb79b5aece0292a0784b406a97e2c91563e17"
Jul 17 05:32:29.558654 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:29.558551    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-4nj99" event={"ID":"5c29f22f-0d63-45e3-ba82-2e851ccbcbae","Type":"ContainerStarted","Data":"5a7e4c03a8b0fdec99983e85464365703860c8477ba107041791b031b52327ee"}
Jul 17 05:32:29.580223 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:29.580170    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-4nj99" podStartSLOduration=19.580154074 podStartE2EDuration="19.580154074s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:32:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:32:29.57521457 +0000 UTC m=+2537.258808256" watchObservedRunningTime="2026-07-17 05:32:29.580154074 +0000 UTC m=+2537.263747731"
Jul 17 05:32:30.568462 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:30.568425    2562 generic.go:356] "Generic (PLEG): container finished" podID="5c29f22f-0d63-45e3-ba82-2e851ccbcbae" containerID="5a7e4c03a8b0fdec99983e85464365703860c8477ba107041791b031b52327ee" exitCode=6
Jul 17 05:32:30.569202 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:30.569175    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-4nj99" event={"ID":"5c29f22f-0d63-45e3-ba82-2e851ccbcbae","Type":"ContainerDied","Data":"5a7e4c03a8b0fdec99983e85464365703860c8477ba107041791b031b52327ee"}
Jul 17 05:32:30.569402 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:30.569378    2562 scope.go:117] "RemoveContainer" containerID="200917ea2f2cbf0f39774cc341bdb79b5aece0292a0784b406a97e2c91563e17"
Jul 17 05:32:30.569944 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:30.569924    2562 scope.go:117] "RemoveContainer" containerID="5a7e4c03a8b0fdec99983e85464365703860c8477ba107041791b031b52327ee"
Jul 17 05:32:30.570389 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:32:30.570362    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-4e221674-4nj99_odh-ai-gateway-infra(5c29f22f-0d63-45e3-ba82-2e851ccbcbae)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-4nj99" podUID="5c29f22f-0d63-45e3-ba82-2e851ccbcbae"
Jul 17 05:32:30.586005 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:30.585979    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-4nj99"]
Jul 17 05:32:31.724692 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:31.724660    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-4nj99"
Jul 17 05:32:31.845521 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:31.845424    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c29f22f-0d63-45e3-ba82-2e851ccbcbae-maas-api-service-ca\") pod \"5c29f22f-0d63-45e3-ba82-2e851ccbcbae\" (UID: \"5c29f22f-0d63-45e3-ba82-2e851ccbcbae\") "
Jul 17 05:32:31.845790 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:31.845767    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c29f22f-0d63-45e3-ba82-2e851ccbcbae-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5c29f22f-0d63-45e3-ba82-2e851ccbcbae" (UID: "5c29f22f-0d63-45e3-ba82-2e851ccbcbae"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:32:31.947004 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:31.946947    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c29f22f-0d63-45e3-ba82-2e851ccbcbae-maas-api-service-ca\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:32:32.584789 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:32.584735    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-4nj99" event={"ID":"5c29f22f-0d63-45e3-ba82-2e851ccbcbae","Type":"ContainerDied","Data":"202a005c811841a6429efd032ba100612e13e7ed31cb59602fdafa4b26639266"}
Jul 17 05:32:32.584789 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:32.584793    2562 scope.go:117] "RemoveContainer" containerID="5a7e4c03a8b0fdec99983e85464365703860c8477ba107041791b031b52327ee"
Jul 17 05:32:32.585096 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:32.584743    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-4nj99"
Jul 17 05:32:32.608643 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:32.608610    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-4nj99"]
Jul 17 05:32:32.611004 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:32.610979    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-4nj99"]
Jul 17 05:32:32.889409 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:32.889331    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c29f22f-0d63-45e3-ba82-2e851ccbcbae" path="/var/lib/kubelet/pods/5c29f22f-0d63-45e3-ba82-2e851ccbcbae/volumes"
Jul 17 05:32:34.492836 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:34.492793    2562 patch_prober.go:28] interesting pod/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 17 05:32:34.493382 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:34.492854    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5" podUID="e875c9d9-5339-4913-9a05-2bc9e6a0fe4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 17 05:32:34.599298 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:34.599266    2562 generic.go:356] "Generic (PLEG): container finished" podID="e875c9d9-5339-4913-9a05-2bc9e6a0fe4d" containerID="a5e4c1bdd8557a5da5d62275218add43a4fb58fe1e0b57ed0d9b6d348ab6eb84" exitCode=0
Jul 17 05:32:34.599448 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:34.599338    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5" event={"ID":"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d","Type":"ContainerDied","Data":"a5e4c1bdd8557a5da5d62275218add43a4fb58fe1e0b57ed0d9b6d348ab6eb84"}
Jul 17 05:32:34.599783 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:34.599767    2562 scope.go:117] "RemoveContainer" containerID="a5e4c1bdd8557a5da5d62275218add43a4fb58fe1e0b57ed0d9b6d348ab6eb84"
Jul 17 05:32:34.600777 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:34.600753    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:32:34.600833 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:34.600814    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:32:34.600870 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:34.600846    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:32:35.606685 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:35.606652    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5" event={"ID":"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d","Type":"ContainerStarted","Data":"e327effe6675c17bbb9c6473be4b3361210511905784555fa0a46335281a6df1"}
Jul 17 05:32:35.891562 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:35.891458    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5"
Jul 17 05:32:35.891747 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:35.891691    2562 patch_prober.go:28] interesting pod/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 17 05:32:35.891815 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:35.891750    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5" podUID="e875c9d9-5339-4913-9a05-2bc9e6a0fe4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 17 05:32:36.891025 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:36.890991    2562 patch_prober.go:28] interesting pod/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 17 05:32:36.891420 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:36.891043    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5" podUID="e875c9d9-5339-4913-9a05-2bc9e6a0fe4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 17 05:32:37.891259 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:37.891221    2562 patch_prober.go:28] interesting pod/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 17 05:32:37.891658 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:37.891272    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5" podUID="e875c9d9-5339-4913-9a05-2bc9e6a0fe4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 17 05:32:38.891125 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:38.891091    2562 patch_prober.go:28] interesting pod/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 17 05:32:38.891316 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:38.891147    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5" podUID="e875c9d9-5339-4913-9a05-2bc9e6a0fe4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 17 05:32:39.895838 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:39.895804    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5"
Jul 17 05:32:39.896287 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:39.896084    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5"
Jul 17 05:32:39.896754 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:39.896737    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5"
Jul 17 05:32:41.994682 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:41.994634    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-857d512b-7567bf77d9-6qq2b"]
Jul 17 05:32:41.995199 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:41.994904    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-857d512b-7567bf77d9-6qq2b" podUID="40c39234-3495-4621-ac2a-76503def4b04" containerName="maas-api" containerID="cri-o://622b41c273f8f19fbb4412bcd3eb0da4575ac67200a78ba7c0ce5306b24a9dad" gracePeriod=30
Jul 17 05:32:42.243402 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:42.243371    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-857d512b-7567bf77d9-6qq2b"
Jul 17 05:32:42.347871 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:42.347838    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40c39234-3495-4621-ac2a-76503def4b04-openshift-service-ca\") pod \"40c39234-3495-4621-ac2a-76503def4b04\" (UID: \"40c39234-3495-4621-ac2a-76503def4b04\") "
Jul 17 05:32:42.348096 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:42.347889    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40c39234-3495-4621-ac2a-76503def4b04-maas-api-tls\") pod \"40c39234-3495-4621-ac2a-76503def4b04\" (UID: \"40c39234-3495-4621-ac2a-76503def4b04\") "
Jul 17 05:32:42.348096 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:42.347911    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xgjs5\" (UniqueName: \"kubernetes.io/projected/40c39234-3495-4621-ac2a-76503def4b04-kube-api-access-xgjs5\") pod \"40c39234-3495-4621-ac2a-76503def4b04\" (UID: \"40c39234-3495-4621-ac2a-76503def4b04\") "
Jul 17 05:32:42.348306 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:42.348280    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40c39234-3495-4621-ac2a-76503def4b04-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "40c39234-3495-4621-ac2a-76503def4b04" (UID: "40c39234-3495-4621-ac2a-76503def4b04"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:32:42.350147 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:42.350118    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40c39234-3495-4621-ac2a-76503def4b04-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "40c39234-3495-4621-ac2a-76503def4b04" (UID: "40c39234-3495-4621-ac2a-76503def4b04"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 05:32:42.350256 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:42.350236    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40c39234-3495-4621-ac2a-76503def4b04-kube-api-access-xgjs5" (OuterVolumeSpecName: "kube-api-access-xgjs5") pod "40c39234-3495-4621-ac2a-76503def4b04" (UID: "40c39234-3495-4621-ac2a-76503def4b04"). InnerVolumeSpecName "kube-api-access-xgjs5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:32:42.449121 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:42.449085    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40c39234-3495-4621-ac2a-76503def4b04-openshift-service-ca\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:32:42.449121 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:42.449118    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40c39234-3495-4621-ac2a-76503def4b04-maas-api-tls\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:32:42.449121 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:42.449131    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgjs5\" (UniqueName: \"kubernetes.io/projected/40c39234-3495-4621-ac2a-76503def4b04-kube-api-access-xgjs5\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:32:42.645034 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:42.644933    2562 generic.go:356] "Generic (PLEG): container finished" podID="40c39234-3495-4621-ac2a-76503def4b04" containerID="622b41c273f8f19fbb4412bcd3eb0da4575ac67200a78ba7c0ce5306b24a9dad" exitCode=0
Jul 17 05:32:42.645034 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:42.645007    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-857d512b-7567bf77d9-6qq2b"
Jul 17 05:32:42.645034 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:42.645023    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-857d512b-7567bf77d9-6qq2b" event={"ID":"40c39234-3495-4621-ac2a-76503def4b04","Type":"ContainerDied","Data":"622b41c273f8f19fbb4412bcd3eb0da4575ac67200a78ba7c0ce5306b24a9dad"}
Jul 17 05:32:42.645277 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:42.645075    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-857d512b-7567bf77d9-6qq2b" event={"ID":"40c39234-3495-4621-ac2a-76503def4b04","Type":"ContainerDied","Data":"0f732dcd0e72b6f4d7e7c47724ac2b726eff5e211ceac57c73d10f71bcfc86f3"}
Jul 17 05:32:42.645277 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:42.645098    2562 scope.go:117] "RemoveContainer" containerID="622b41c273f8f19fbb4412bcd3eb0da4575ac67200a78ba7c0ce5306b24a9dad"
Jul 17 05:32:42.654344 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:42.654326    2562 scope.go:117] "RemoveContainer" containerID="622b41c273f8f19fbb4412bcd3eb0da4575ac67200a78ba7c0ce5306b24a9dad"
Jul 17 05:32:42.654625 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:32:42.654606    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"622b41c273f8f19fbb4412bcd3eb0da4575ac67200a78ba7c0ce5306b24a9dad\": container with ID starting with 622b41c273f8f19fbb4412bcd3eb0da4575ac67200a78ba7c0ce5306b24a9dad not found: ID does not exist" containerID="622b41c273f8f19fbb4412bcd3eb0da4575ac67200a78ba7c0ce5306b24a9dad"
Jul 17 05:32:42.654673 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:42.654634    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"622b41c273f8f19fbb4412bcd3eb0da4575ac67200a78ba7c0ce5306b24a9dad"} err="failed to get container status \"622b41c273f8f19fbb4412bcd3eb0da4575ac67200a78ba7c0ce5306b24a9dad\": rpc error: code = NotFound desc = could not find container \"622b41c273f8f19fbb4412bcd3eb0da4575ac67200a78ba7c0ce5306b24a9dad\": container with ID starting with 622b41c273f8f19fbb4412bcd3eb0da4575ac67200a78ba7c0ce5306b24a9dad not found: ID does not exist"
Jul 17 05:32:42.666564 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:42.666531    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-857d512b-7567bf77d9-6qq2b"]
Jul 17 05:32:42.670836 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:42.670810    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-857d512b-7567bf77d9-6qq2b"]
Jul 17 05:32:42.885804 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:42.885769    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40c39234-3495-4621-ac2a-76503def4b04" path="/var/lib/kubelet/pods/40c39234-3495-4621-ac2a-76503def4b04/volumes"
Jul 17 05:32:57.285987 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:57.285807    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5"]
Jul 17 05:32:57.286469 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:32:57.286082    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5" podUID="e875c9d9-5339-4913-9a05-2bc9e6a0fe4d" containerName="istio-proxy" containerID="cri-o://e327effe6675c17bbb9c6473be4b3361210511905784555fa0a46335281a6df1" gracePeriod=30
Jul 17 05:33:02.544641 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:02.544615    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5"
Jul 17 05:33:02.646387 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:02.646348    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-istio-data\") pod \"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d\" (UID: \"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d\") "
Jul 17 05:33:02.646387 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:02.646390    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jhhjl\" (UniqueName: \"kubernetes.io/projected/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-kube-api-access-jhhjl\") pod \"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d\" (UID: \"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d\") "
Jul 17 05:33:02.646632 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:02.646418    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-istio-podinfo\") pod \"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d\" (UID: \"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d\") "
Jul 17 05:33:02.646632 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:02.646439    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-istio-token\") pod \"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d\" (UID: \"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d\") "
Jul 17 05:33:02.646632 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:02.646483    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-istio-envoy\") pod \"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d\" (UID: \"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d\") "
Jul 17 05:33:02.646632 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:02.646558    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-istiod-ca-cert\") pod \"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d\" (UID: \"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d\") "
Jul 17 05:33:02.646632 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:02.646621    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-workload-certs\") pod \"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d\" (UID: \"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d\") "
Jul 17 05:33:02.646897 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:02.646681    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-credential-socket\") pod \"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d\" (UID: \"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d\") "
Jul 17 05:33:02.646897 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:02.646725    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-workload-socket\") pod \"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d\" (UID: \"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d\") "
Jul 17 05:33:02.647086 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:02.647054    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e875c9d9-5339-4913-9a05-2bc9e6a0fe4d" (UID: "e875c9d9-5339-4913-9a05-2bc9e6a0fe4d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:33:02.647145 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:02.647054    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e875c9d9-5339-4913-9a05-2bc9e6a0fe4d" (UID: "e875c9d9-5339-4913-9a05-2bc9e6a0fe4d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:33:02.647301 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:02.647255    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e875c9d9-5339-4913-9a05-2bc9e6a0fe4d" (UID: "e875c9d9-5339-4913-9a05-2bc9e6a0fe4d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:33:02.647301 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:02.647270    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e875c9d9-5339-4913-9a05-2bc9e6a0fe4d" (UID: "e875c9d9-5339-4913-9a05-2bc9e6a0fe4d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:33:02.647301 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:02.647228    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-istio-data" (OuterVolumeSpecName: "istio-data") pod "e875c9d9-5339-4913-9a05-2bc9e6a0fe4d" (UID: "e875c9d9-5339-4913-9a05-2bc9e6a0fe4d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:33:02.648947 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:02.648916    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e875c9d9-5339-4913-9a05-2bc9e6a0fe4d" (UID: "e875c9d9-5339-4913-9a05-2bc9e6a0fe4d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:33:02.649238 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:02.649220    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-istio-token" (OuterVolumeSpecName: "istio-token") pod "e875c9d9-5339-4913-9a05-2bc9e6a0fe4d" (UID: "e875c9d9-5339-4913-9a05-2bc9e6a0fe4d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:33:02.649340 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:02.649321    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e875c9d9-5339-4913-9a05-2bc9e6a0fe4d" (UID: "e875c9d9-5339-4913-9a05-2bc9e6a0fe4d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 05:33:02.649394 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:02.649361    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-kube-api-access-jhhjl" (OuterVolumeSpecName: "kube-api-access-jhhjl") pod "e875c9d9-5339-4913-9a05-2bc9e6a0fe4d" (UID: "e875c9d9-5339-4913-9a05-2bc9e6a0fe4d"). InnerVolumeSpecName "kube-api-access-jhhjl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:33:02.747555 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:02.747522    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-credential-socket\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:33:02.747555 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:02.747549    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-workload-socket\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:33:02.747555 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:02.747558    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-istio-data\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:33:02.747864 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:02.747567    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhhjl\" (UniqueName: \"kubernetes.io/projected/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-kube-api-access-jhhjl\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:33:02.747864 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:02.747575    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-istio-podinfo\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:33:02.747864 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:02.747583    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-istio-token\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:33:02.747864 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:02.747590    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-istio-envoy\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:33:02.747864 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:02.747597    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-istiod-ca-cert\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:33:02.747864 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:02.747604    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d-workload-certs\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:33:02.750357 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:02.750322    2562 generic.go:356] "Generic (PLEG): container finished" podID="e875c9d9-5339-4913-9a05-2bc9e6a0fe4d" containerID="e327effe6675c17bbb9c6473be4b3361210511905784555fa0a46335281a6df1" exitCode=0
Jul 17 05:33:02.750487 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:02.750372    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5"
Jul 17 05:33:02.750487 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:02.750413    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5" event={"ID":"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d","Type":"ContainerDied","Data":"e327effe6675c17bbb9c6473be4b3361210511905784555fa0a46335281a6df1"}
Jul 17 05:33:02.750487 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:02.750471    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5" event={"ID":"e875c9d9-5339-4913-9a05-2bc9e6a0fe4d","Type":"ContainerDied","Data":"c5d8883f316e95f840cd76016df0841cda9ccfdd2f27bf05d72edb30664811b3"}
Jul 17 05:33:02.750633 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:02.750494    2562 scope.go:117] "RemoveContainer" containerID="e327effe6675c17bbb9c6473be4b3361210511905784555fa0a46335281a6df1"
Jul 17 05:33:02.760448 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:02.760432    2562 scope.go:117] "RemoveContainer" containerID="a5e4c1bdd8557a5da5d62275218add43a4fb58fe1e0b57ed0d9b6d348ab6eb84"
Jul 17 05:33:02.768910 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:02.768892    2562 scope.go:117] "RemoveContainer" containerID="e327effe6675c17bbb9c6473be4b3361210511905784555fa0a46335281a6df1"
Jul 17 05:33:02.769268 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:33:02.769245    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e327effe6675c17bbb9c6473be4b3361210511905784555fa0a46335281a6df1\": container with ID starting with e327effe6675c17bbb9c6473be4b3361210511905784555fa0a46335281a6df1 not found: ID does not exist" containerID="e327effe6675c17bbb9c6473be4b3361210511905784555fa0a46335281a6df1"
Jul 17 05:33:02.769363 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:02.769278    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e327effe6675c17bbb9c6473be4b3361210511905784555fa0a46335281a6df1"} err="failed to get container status \"e327effe6675c17bbb9c6473be4b3361210511905784555fa0a46335281a6df1\": rpc error: code = NotFound desc = could not find container \"e327effe6675c17bbb9c6473be4b3361210511905784555fa0a46335281a6df1\": container with ID starting with e327effe6675c17bbb9c6473be4b3361210511905784555fa0a46335281a6df1 not found: ID does not exist"
Jul 17 05:33:02.769363 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:02.769295    2562 scope.go:117] "RemoveContainer" containerID="a5e4c1bdd8557a5da5d62275218add43a4fb58fe1e0b57ed0d9b6d348ab6eb84"
Jul 17 05:33:02.769571 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:33:02.769554    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5e4c1bdd8557a5da5d62275218add43a4fb58fe1e0b57ed0d9b6d348ab6eb84\": container with ID starting with a5e4c1bdd8557a5da5d62275218add43a4fb58fe1e0b57ed0d9b6d348ab6eb84 not found: ID does not exist" containerID="a5e4c1bdd8557a5da5d62275218add43a4fb58fe1e0b57ed0d9b6d348ab6eb84"
Jul 17 05:33:02.769615 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:02.769578    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5e4c1bdd8557a5da5d62275218add43a4fb58fe1e0b57ed0d9b6d348ab6eb84"} err="failed to get container status \"a5e4c1bdd8557a5da5d62275218add43a4fb58fe1e0b57ed0d9b6d348ab6eb84\": rpc error: code = NotFound desc = could not find container \"a5e4c1bdd8557a5da5d62275218add43a4fb58fe1e0b57ed0d9b6d348ab6eb84\": container with ID starting with a5e4c1bdd8557a5da5d62275218add43a4fb58fe1e0b57ed0d9b6d348ab6eb84 not found: ID does not exist"
Jul 17 05:33:02.772800 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:02.772776    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5"]
Jul 17 05:33:02.778187 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:02.778161    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-857d512b-openshift-default-59988bbf67-t4jn5"]
Jul 17 05:33:02.884223 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:02.884188    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e875c9d9-5339-4913-9a05-2bc9e6a0fe4d" path="/var/lib/kubelet/pods/e875c9d9-5339-4913-9a05-2bc9e6a0fe4d/volumes"
Jul 17 05:33:10.884341 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:10.884312    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-29ngv"]
Jul 17 05:33:10.884727 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:10.884664    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7df963d-37d1-4f83-afc9-053e37e0c5eb" containerName="revoke-keys"
Jul 17 05:33:10.884727 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:10.884673    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7df963d-37d1-4f83-afc9-053e37e0c5eb" containerName="revoke-keys"
Jul 17 05:33:10.884727 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:10.884687    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40c39234-3495-4621-ac2a-76503def4b04" containerName="maas-api"
Jul 17 05:33:10.884727 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:10.884693    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="40c39234-3495-4621-ac2a-76503def4b04" containerName="maas-api"
Jul 17 05:33:10.884727 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:10.884702    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c29f22f-0d63-45e3-ba82-2e851ccbcbae" containerName="revoke-keys"
Jul 17 05:33:10.884727 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:10.884708    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c29f22f-0d63-45e3-ba82-2e851ccbcbae" containerName="revoke-keys"
Jul 17 05:33:10.884727 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:10.884717    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e875c9d9-5339-4913-9a05-2bc9e6a0fe4d" containerName="istio-proxy"
Jul 17 05:33:10.884727 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:10.884723    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e875c9d9-5339-4913-9a05-2bc9e6a0fe4d" containerName="istio-proxy"
Jul 17 05:33:10.885002 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:10.884738    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7df963d-37d1-4f83-afc9-053e37e0c5eb" containerName="revoke-keys"
Jul 17 05:33:10.885002 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:10.884743    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7df963d-37d1-4f83-afc9-053e37e0c5eb" containerName="revoke-keys"
Jul 17 05:33:10.885002 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:10.884751    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c29f22f-0d63-45e3-ba82-2e851ccbcbae" containerName="revoke-keys"
Jul 17 05:33:10.885002 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:10.884756    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c29f22f-0d63-45e3-ba82-2e851ccbcbae" containerName="revoke-keys"
Jul 17 05:33:10.885002 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:10.884811    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="40c39234-3495-4621-ac2a-76503def4b04" containerName="maas-api"
Jul 17 05:33:10.885002 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:10.884820    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7df963d-37d1-4f83-afc9-053e37e0c5eb" containerName="revoke-keys"
Jul 17 05:33:10.885002 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:10.884826    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7df963d-37d1-4f83-afc9-053e37e0c5eb" containerName="revoke-keys"
Jul 17 05:33:10.885002 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:10.884832    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e875c9d9-5339-4913-9a05-2bc9e6a0fe4d" containerName="istio-proxy"
Jul 17 05:33:10.885002 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:10.884838    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e875c9d9-5339-4913-9a05-2bc9e6a0fe4d" containerName="istio-proxy"
Jul 17 05:33:10.885002 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:10.884843    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c29f22f-0d63-45e3-ba82-2e851ccbcbae" containerName="revoke-keys"
Jul 17 05:33:10.885002 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:10.884848    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c29f22f-0d63-45e3-ba82-2e851ccbcbae" containerName="revoke-keys"
Jul 17 05:33:10.888018 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:10.888000    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-29ngv"
Jul 17 05:33:10.890841 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:10.890812    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qh8zj\""
Jul 17 05:33:10.893540 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:10.893507    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-29ngv"]
Jul 17 05:33:11.028915 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:11.028872    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a4dde0d-6c33-4847-a4a4-806a0da369aa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4e221674-29ngv\" (UID: \"7a4dde0d-6c33-4847-a4a4-806a0da369aa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-29ngv"
Jul 17 05:33:11.129942 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:11.129908    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a4dde0d-6c33-4847-a4a4-806a0da369aa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4e221674-29ngv\" (UID: \"7a4dde0d-6c33-4847-a4a4-806a0da369aa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-29ngv"
Jul 17 05:33:11.130534 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:11.130514    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a4dde0d-6c33-4847-a4a4-806a0da369aa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4e221674-29ngv\" (UID: \"7a4dde0d-6c33-4847-a4a4-806a0da369aa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-29ngv"
Jul 17 05:33:11.200303 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:11.200213    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-29ngv"
Jul 17 05:33:11.334456 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:11.334434    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-29ngv"]
Jul 17 05:33:11.337107 ip-10-0-132-19 kubenswrapper[2562]: W0717 05:33:11.337081    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a4dde0d_6c33_4847_a4a4_806a0da369aa.slice/crio-9f0ce03eb51a439bc0aff06bd2f06346712309f2993b4178ed0d03610ab5ba0e WatchSource:0}: Error finding container 9f0ce03eb51a439bc0aff06bd2f06346712309f2993b4178ed0d03610ab5ba0e: Status 404 returned error can't find the container with id 9f0ce03eb51a439bc0aff06bd2f06346712309f2993b4178ed0d03610ab5ba0e
Jul 17 05:33:11.797361 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:11.797318    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-29ngv" event={"ID":"7a4dde0d-6c33-4847-a4a4-806a0da369aa","Type":"ContainerStarted","Data":"d84e9fd307a78f032782eab3e9d6626b656a4468dff88a79d00b6f3106f56e93"}
Jul 17 05:33:11.797708 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:11.797384    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-29ngv" event={"ID":"7a4dde0d-6c33-4847-a4a4-806a0da369aa","Type":"ContainerStarted","Data":"9f0ce03eb51a439bc0aff06bd2f06346712309f2993b4178ed0d03610ab5ba0e"}
Jul 17 05:33:11.814693 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:11.814645    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-29ngv" podStartSLOduration=1.8146289279999999 podStartE2EDuration="1.814628928s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:33:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:33:11.81268862 +0000 UTC m=+2579.496282278" watchObservedRunningTime="2026-07-17 05:33:11.814628928 +0000 UTC m=+2579.498222632"
Jul 17 05:33:12.803494 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:12.803454    2562 generic.go:356] "Generic (PLEG): container finished" podID="7a4dde0d-6c33-4847-a4a4-806a0da369aa" containerID="d84e9fd307a78f032782eab3e9d6626b656a4468dff88a79d00b6f3106f56e93" exitCode=6
Jul 17 05:33:12.803863 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:12.803539    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-29ngv" event={"ID":"7a4dde0d-6c33-4847-a4a4-806a0da369aa","Type":"ContainerDied","Data":"d84e9fd307a78f032782eab3e9d6626b656a4468dff88a79d00b6f3106f56e93"}
Jul 17 05:33:12.803863 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:12.803693    2562 scope.go:117] "RemoveContainer" containerID="d84e9fd307a78f032782eab3e9d6626b656a4468dff88a79d00b6f3106f56e93"
Jul 17 05:33:13.810232 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:13.810149    2562 generic.go:356] "Generic (PLEG): container finished" podID="7a4dde0d-6c33-4847-a4a4-806a0da369aa" containerID="f45e28e1f11e3a7866c7541c613aeb731b5958521bff0008445d2760c85d3eee" exitCode=6
Jul 17 05:33:13.810641 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:13.810236    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-29ngv" event={"ID":"7a4dde0d-6c33-4847-a4a4-806a0da369aa","Type":"ContainerDied","Data":"f45e28e1f11e3a7866c7541c613aeb731b5958521bff0008445d2760c85d3eee"}
Jul 17 05:33:13.810641 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:13.810293    2562 scope.go:117] "RemoveContainer" containerID="d84e9fd307a78f032782eab3e9d6626b656a4468dff88a79d00b6f3106f56e93"
Jul 17 05:33:13.810641 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:13.810502    2562 scope.go:117] "RemoveContainer" containerID="f45e28e1f11e3a7866c7541c613aeb731b5958521bff0008445d2760c85d3eee"
Jul 17 05:33:13.810766 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:33:13.810732    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-4e221674-29ngv_odh-ai-gateway-infra(7a4dde0d-6c33-4847-a4a4-806a0da369aa)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-29ngv" podUID="7a4dde0d-6c33-4847-a4a4-806a0da369aa"
Jul 17 05:33:14.816802 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:14.816773    2562 scope.go:117] "RemoveContainer" containerID="f45e28e1f11e3a7866c7541c613aeb731b5958521bff0008445d2760c85d3eee"
Jul 17 05:33:14.817213 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:33:14.817011    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-4e221674-29ngv_odh-ai-gateway-infra(7a4dde0d-6c33-4847-a4a4-806a0da369aa)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-29ngv" podUID="7a4dde0d-6c33-4847-a4a4-806a0da369aa"
Jul 17 05:33:17.210684 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.210653    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh"]
Jul 17 05:33:17.211107 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.211059    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7df963d-37d1-4f83-afc9-053e37e0c5eb" containerName="revoke-keys"
Jul 17 05:33:17.211107 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.211070    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7df963d-37d1-4f83-afc9-053e37e0c5eb" containerName="revoke-keys"
Jul 17 05:33:17.211107 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.211086    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c29f22f-0d63-45e3-ba82-2e851ccbcbae" containerName="revoke-keys"
Jul 17 05:33:17.211107 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.211091    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c29f22f-0d63-45e3-ba82-2e851ccbcbae" containerName="revoke-keys"
Jul 17 05:33:17.211107 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.211100    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e875c9d9-5339-4913-9a05-2bc9e6a0fe4d" containerName="istio-proxy"
Jul 17 05:33:17.211107 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.211106    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e875c9d9-5339-4913-9a05-2bc9e6a0fe4d" containerName="istio-proxy"
Jul 17 05:33:17.211335 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.211202    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c29f22f-0d63-45e3-ba82-2e851ccbcbae" containerName="revoke-keys"
Jul 17 05:33:17.211335 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.211214    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7df963d-37d1-4f83-afc9-053e37e0c5eb" containerName="revoke-keys"
Jul 17 05:33:17.214026 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.214008    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh"
Jul 17 05:33:17.216849 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.216832    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-e73f00c0-openshift-default-dockercfg-g4t4q\""
Jul 17 05:33:17.232019 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.231955    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh"]
Jul 17 05:33:17.293242 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.293200    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-istio-envoy\") pod \"e2e-mt-e73f00c0-openshift-default-6869488947-jgffh\" (UID: \"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723\") " pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh"
Jul 17 05:33:17.293560 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.293534    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-istio-data\") pod \"e2e-mt-e73f00c0-openshift-default-6869488947-jgffh\" (UID: \"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723\") " pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh"
Jul 17 05:33:17.293787 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.293709    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-istio-token\") pod \"e2e-mt-e73f00c0-openshift-default-6869488947-jgffh\" (UID: \"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723\") " pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh"
Jul 17 05:33:17.293787 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.293762    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-istiod-ca-cert\") pod \"e2e-mt-e73f00c0-openshift-default-6869488947-jgffh\" (UID: \"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723\") " pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh"
Jul 17 05:33:17.293985 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.293850    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-istio-podinfo\") pod \"e2e-mt-e73f00c0-openshift-default-6869488947-jgffh\" (UID: \"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723\") " pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh"
Jul 17 05:33:17.293985 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.293918    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-workload-certs\") pod \"e2e-mt-e73f00c0-openshift-default-6869488947-jgffh\" (UID: \"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723\") " pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh"
Jul 17 05:33:17.294114 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.294030    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-workload-socket\") pod \"e2e-mt-e73f00c0-openshift-default-6869488947-jgffh\" (UID: \"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723\") " pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh"
Jul 17 05:33:17.294114 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.294062    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6n4r7\" (UniqueName: \"kubernetes.io/projected/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-kube-api-access-6n4r7\") pod \"e2e-mt-e73f00c0-openshift-default-6869488947-jgffh\" (UID: \"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723\") " pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh"
Jul 17 05:33:17.294242 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.294155    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-credential-socket\") pod \"e2e-mt-e73f00c0-openshift-default-6869488947-jgffh\" (UID: \"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723\") " pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh"
Jul 17 05:33:17.395329 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.395293    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-istiod-ca-cert\") pod \"e2e-mt-e73f00c0-openshift-default-6869488947-jgffh\" (UID: \"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723\") " pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh"
Jul 17 05:33:17.395576 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.395352    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-istio-podinfo\") pod \"e2e-mt-e73f00c0-openshift-default-6869488947-jgffh\" (UID: \"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723\") " pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh"
Jul 17 05:33:17.395576 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.395383    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-workload-certs\") pod \"e2e-mt-e73f00c0-openshift-default-6869488947-jgffh\" (UID: \"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723\") " pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh"
Jul 17 05:33:17.395576 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.395407    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-workload-socket\") pod \"e2e-mt-e73f00c0-openshift-default-6869488947-jgffh\" (UID: \"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723\") " pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh"
Jul 17 05:33:17.395576 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.395425    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6n4r7\" (UniqueName: \"kubernetes.io/projected/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-kube-api-access-6n4r7\") pod \"e2e-mt-e73f00c0-openshift-default-6869488947-jgffh\" (UID: \"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723\") " pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh"
Jul 17 05:33:17.395576 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.395454    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-credential-socket\") pod \"e2e-mt-e73f00c0-openshift-default-6869488947-jgffh\" (UID: \"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723\") " pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh"
Jul 17 05:33:17.395576 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.395485    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-istio-envoy\") pod \"e2e-mt-e73f00c0-openshift-default-6869488947-jgffh\" (UID: \"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723\") " pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh"
Jul 17 05:33:17.395939 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.395631    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-istio-data\") pod \"e2e-mt-e73f00c0-openshift-default-6869488947-jgffh\" (UID: \"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723\") " pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh"
Jul 17 05:33:17.395939 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.395659    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-istio-token\") pod \"e2e-mt-e73f00c0-openshift-default-6869488947-jgffh\" (UID: \"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723\") " pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh"
Jul 17 05:33:17.395939 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.395857    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-workload-socket\") pod \"e2e-mt-e73f00c0-openshift-default-6869488947-jgffh\" (UID: \"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723\") " pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh"
Jul 17 05:33:17.396129 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.396008    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-workload-certs\") pod \"e2e-mt-e73f00c0-openshift-default-6869488947-jgffh\" (UID: \"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723\") " pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh"
Jul 17 05:33:17.396129 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.396031    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-credential-socket\") pod \"e2e-mt-e73f00c0-openshift-default-6869488947-jgffh\" (UID: \"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723\") " pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh"
Jul 17 05:33:17.396129 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.396091    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-istio-data\") pod \"e2e-mt-e73f00c0-openshift-default-6869488947-jgffh\" (UID: \"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723\") " pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh"
Jul 17 05:33:17.396262 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.396175    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-istiod-ca-cert\") pod \"e2e-mt-e73f00c0-openshift-default-6869488947-jgffh\" (UID: \"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723\") " pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh"
Jul 17 05:33:17.398029 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.398006    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-istio-envoy\") pod \"e2e-mt-e73f00c0-openshift-default-6869488947-jgffh\" (UID: \"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723\") " pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh"
Jul 17 05:33:17.398251 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.398235    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-istio-podinfo\") pod \"e2e-mt-e73f00c0-openshift-default-6869488947-jgffh\" (UID: \"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723\") " pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh"
Jul 17 05:33:17.404901 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.404881    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-istio-token\") pod \"e2e-mt-e73f00c0-openshift-default-6869488947-jgffh\" (UID: \"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723\") " pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh"
Jul 17 05:33:17.405893 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.405870    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6n4r7\" (UniqueName: \"kubernetes.io/projected/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-kube-api-access-6n4r7\") pod \"e2e-mt-e73f00c0-openshift-default-6869488947-jgffh\" (UID: \"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723\") " pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh"
Jul 17 05:33:17.525192 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.525105    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh"
Jul 17 05:33:17.674204 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.673062    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh"]
Jul 17 05:33:17.679946 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.679900    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:33:17.680080 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.679999    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:33:17.680080 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.680037    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:33:17.834184 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.834136    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh" event={"ID":"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723","Type":"ContainerStarted","Data":"e560d5d7acb93e34ff1108b02905b697455493e751d771ae1b1901e5b70efbdb"}
Jul 17 05:33:17.834358 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.834209    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh" event={"ID":"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723","Type":"ContainerStarted","Data":"431608bff2aad862b1261bbe9b13a0f85cc2c1822ed39a617fefb248916cc95e"}
Jul 17 05:33:17.858103 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:17.858046    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh" podStartSLOduration=0.858025306 podStartE2EDuration="858.025306ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:33:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:33:17.857645983 +0000 UTC m=+2585.541239667" watchObservedRunningTime="2026-07-17 05:33:17.858025306 +0000 UTC m=+2585.541618964"
Jul 17 05:33:18.525941 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:18.525899    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh"
Jul 17 05:33:18.531066 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:18.531041    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh"
Jul 17 05:33:18.839114 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:18.839083    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh"
Jul 17 05:33:18.840192 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:18.840170    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh"
Jul 17 05:33:19.976133 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:19.976088    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e73f00c0-9d866c87c-5rqhw"]
Jul 17 05:33:19.978955 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:19.978931    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e73f00c0-9d866c87c-5rqhw"
Jul 17 05:33:19.982121 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:19.982096    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-e73f00c0-serving-cert\""
Jul 17 05:33:19.984061 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:19.984036    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-xwcd7\""
Jul 17 05:33:20.002977 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:20.002923    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e73f00c0-9d866c87c-5rqhw"]
Jul 17 05:33:20.120014 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:20.119956    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e61eebe1-0c12-4b0b-8669-692bfe8a9e8c-openshift-service-ca\") pod \"maas-api-e2e-mt-e73f00c0-9d866c87c-5rqhw\" (UID: \"e61eebe1-0c12-4b0b-8669-692bfe8a9e8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e73f00c0-9d866c87c-5rqhw"
Jul 17 05:33:20.120214 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:20.120070    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e61eebe1-0c12-4b0b-8669-692bfe8a9e8c-maas-api-tls\") pod \"maas-api-e2e-mt-e73f00c0-9d866c87c-5rqhw\" (UID: \"e61eebe1-0c12-4b0b-8669-692bfe8a9e8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e73f00c0-9d866c87c-5rqhw"
Jul 17 05:33:20.120214 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:20.120175    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2bdw\" (UniqueName: \"kubernetes.io/projected/e61eebe1-0c12-4b0b-8669-692bfe8a9e8c-kube-api-access-p2bdw\") pod \"maas-api-e2e-mt-e73f00c0-9d866c87c-5rqhw\" (UID: \"e61eebe1-0c12-4b0b-8669-692bfe8a9e8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e73f00c0-9d866c87c-5rqhw"
Jul 17 05:33:20.220811 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:20.220773    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e61eebe1-0c12-4b0b-8669-692bfe8a9e8c-openshift-service-ca\") pod \"maas-api-e2e-mt-e73f00c0-9d866c87c-5rqhw\" (UID: \"e61eebe1-0c12-4b0b-8669-692bfe8a9e8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e73f00c0-9d866c87c-5rqhw"
Jul 17 05:33:20.220811 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:20.220817    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e61eebe1-0c12-4b0b-8669-692bfe8a9e8c-maas-api-tls\") pod \"maas-api-e2e-mt-e73f00c0-9d866c87c-5rqhw\" (UID: \"e61eebe1-0c12-4b0b-8669-692bfe8a9e8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e73f00c0-9d866c87c-5rqhw"
Jul 17 05:33:20.221080 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:20.220870    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2bdw\" (UniqueName: \"kubernetes.io/projected/e61eebe1-0c12-4b0b-8669-692bfe8a9e8c-kube-api-access-p2bdw\") pod \"maas-api-e2e-mt-e73f00c0-9d866c87c-5rqhw\" (UID: \"e61eebe1-0c12-4b0b-8669-692bfe8a9e8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e73f00c0-9d866c87c-5rqhw"
Jul 17 05:33:20.221080 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:33:20.221003    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-e73f00c0-serving-cert: secret "maas-api-e2e-mt-e73f00c0-serving-cert" not found
Jul 17 05:33:20.221170 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:33:20.221084    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e61eebe1-0c12-4b0b-8669-692bfe8a9e8c-maas-api-tls podName:e61eebe1-0c12-4b0b-8669-692bfe8a9e8c nodeName:}" failed. No retries permitted until 2026-07-17 05:33:20.721065282 +0000 UTC m=+2588.404658917 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e61eebe1-0c12-4b0b-8669-692bfe8a9e8c-maas-api-tls") pod "maas-api-e2e-mt-e73f00c0-9d866c87c-5rqhw" (UID: "e61eebe1-0c12-4b0b-8669-692bfe8a9e8c") : secret "maas-api-e2e-mt-e73f00c0-serving-cert" not found
Jul 17 05:33:20.221477 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:20.221457    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e61eebe1-0c12-4b0b-8669-692bfe8a9e8c-openshift-service-ca\") pod \"maas-api-e2e-mt-e73f00c0-9d866c87c-5rqhw\" (UID: \"e61eebe1-0c12-4b0b-8669-692bfe8a9e8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e73f00c0-9d866c87c-5rqhw"
Jul 17 05:33:20.230173 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:20.230092    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2bdw\" (UniqueName: \"kubernetes.io/projected/e61eebe1-0c12-4b0b-8669-692bfe8a9e8c-kube-api-access-p2bdw\") pod \"maas-api-e2e-mt-e73f00c0-9d866c87c-5rqhw\" (UID: \"e61eebe1-0c12-4b0b-8669-692bfe8a9e8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e73f00c0-9d866c87c-5rqhw"
Jul 17 05:33:20.726126 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:20.726093    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e61eebe1-0c12-4b0b-8669-692bfe8a9e8c-maas-api-tls\") pod \"maas-api-e2e-mt-e73f00c0-9d866c87c-5rqhw\" (UID: \"e61eebe1-0c12-4b0b-8669-692bfe8a9e8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e73f00c0-9d866c87c-5rqhw"
Jul 17 05:33:20.728802 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:20.728776    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e61eebe1-0c12-4b0b-8669-692bfe8a9e8c-maas-api-tls\") pod \"maas-api-e2e-mt-e73f00c0-9d866c87c-5rqhw\" (UID: \"e61eebe1-0c12-4b0b-8669-692bfe8a9e8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e73f00c0-9d866c87c-5rqhw"
Jul 17 05:33:20.891886 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:20.891846    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e73f00c0-9d866c87c-5rqhw"
Jul 17 05:33:21.040151 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:21.040128    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e73f00c0-9d866c87c-5rqhw"]
Jul 17 05:33:21.042744 ip-10-0-132-19 kubenswrapper[2562]: W0717 05:33:21.042705    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode61eebe1_0c12_4b0b_8669_692bfe8a9e8c.slice/crio-9928fc684c683f3a8a81a743b73a9dc38141c79b7f5616bd36f0a177ba0e7f95 WatchSource:0}: Error finding container 9928fc684c683f3a8a81a743b73a9dc38141c79b7f5616bd36f0a177ba0e7f95: Status 404 returned error can't find the container with id 9928fc684c683f3a8a81a743b73a9dc38141c79b7f5616bd36f0a177ba0e7f95
Jul 17 05:33:21.856685 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:21.856587    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e73f00c0-9d866c87c-5rqhw" event={"ID":"e61eebe1-0c12-4b0b-8669-692bfe8a9e8c","Type":"ContainerStarted","Data":"871064e8860039d79319f33ed87ca13b59742e99ea93afa1dcd96c72425cd440"}
Jul 17 05:33:21.856685 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:21.856642    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e73f00c0-9d866c87c-5rqhw" event={"ID":"e61eebe1-0c12-4b0b-8669-692bfe8a9e8c","Type":"ContainerStarted","Data":"9928fc684c683f3a8a81a743b73a9dc38141c79b7f5616bd36f0a177ba0e7f95"}
Jul 17 05:33:21.856896 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:21.856708    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e73f00c0-9d866c87c-5rqhw"
Jul 17 05:33:21.874653 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:21.874605    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e73f00c0-9d866c87c-5rqhw" podStartSLOduration=2.328357326 podStartE2EDuration="2.874588365s" totalImagesPullingTime="546.231039ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:33:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:33:21.873353457 +0000 UTC m=+2589.556947163" watchObservedRunningTime="2026-07-17 05:33:21.874588365 +0000 UTC m=+2589.558182021"
Jul 17 05:33:27.868184 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:27.868155    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e73f00c0-9d866c87c-5rqhw"
Jul 17 05:33:28.884882 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:28.884854    2562 scope.go:117] "RemoveContainer" containerID="f45e28e1f11e3a7866c7541c613aeb731b5958521bff0008445d2760c85d3eee"
Jul 17 05:33:29.899761 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:29.899726    2562 generic.go:356] "Generic (PLEG): container finished" podID="7a4dde0d-6c33-4847-a4a4-806a0da369aa" containerID="df91f7a296835b0cd3d845b5574d82eb3645515f8509b9b6fe28710c7b3ecd75" exitCode=6
Jul 17 05:33:29.900271 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:29.899804    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-29ngv" event={"ID":"7a4dde0d-6c33-4847-a4a4-806a0da369aa","Type":"ContainerDied","Data":"df91f7a296835b0cd3d845b5574d82eb3645515f8509b9b6fe28710c7b3ecd75"}
Jul 17 05:33:29.900271 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:29.899885    2562 scope.go:117] "RemoveContainer" containerID="f45e28e1f11e3a7866c7541c613aeb731b5958521bff0008445d2760c85d3eee"
Jul 17 05:33:29.900271 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:29.900232    2562 scope.go:117] "RemoveContainer" containerID="df91f7a296835b0cd3d845b5574d82eb3645515f8509b9b6fe28710c7b3ecd75"
Jul 17 05:33:29.900483 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:33:29.900462    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-4e221674-29ngv_odh-ai-gateway-infra(7a4dde0d-6c33-4847-a4a4-806a0da369aa)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-29ngv" podUID="7a4dde0d-6c33-4847-a4a4-806a0da369aa"
Jul 17 05:33:30.924402 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:30.924369    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-29ngv"]
Jul 17 05:33:31.105321 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:31.105297    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-29ngv"
Jul 17 05:33:31.136666 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:31.136634    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a4dde0d-6c33-4847-a4a4-806a0da369aa-maas-api-service-ca\") pod \"7a4dde0d-6c33-4847-a4a4-806a0da369aa\" (UID: \"7a4dde0d-6c33-4847-a4a4-806a0da369aa\") "
Jul 17 05:33:31.137037 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:31.137009    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a4dde0d-6c33-4847-a4a4-806a0da369aa-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7a4dde0d-6c33-4847-a4a4-806a0da369aa" (UID: "7a4dde0d-6c33-4847-a4a4-806a0da369aa"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:33:31.237723 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:31.237621    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a4dde0d-6c33-4847-a4a4-806a0da369aa-maas-api-service-ca\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:33:31.913087 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:31.913057    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-29ngv"
Jul 17 05:33:31.913087 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:31.913067    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-29ngv" event={"ID":"7a4dde0d-6c33-4847-a4a4-806a0da369aa","Type":"ContainerDied","Data":"9f0ce03eb51a439bc0aff06bd2f06346712309f2993b4178ed0d03610ab5ba0e"}
Jul 17 05:33:31.913311 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:31.913122    2562 scope.go:117] "RemoveContainer" containerID="df91f7a296835b0cd3d845b5574d82eb3645515f8509b9b6fe28710c7b3ecd75"
Jul 17 05:33:31.937151 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:31.937124    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-29ngv"]
Jul 17 05:33:31.942195 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:31.942171    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-29ngv"]
Jul 17 05:33:32.884306 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:32.884270    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a4dde0d-6c33-4847-a4a4-806a0da369aa" path="/var/lib/kubelet/pods/7a4dde0d-6c33-4847-a4a4-806a0da369aa/volumes"
Jul 17 05:33:34.882160 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:34.882116    2562 patch_prober.go:28] interesting pod/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 17 05:33:34.882834 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:34.882178    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh" podUID="8a4b7bdf-dd83-41a6-bcd1-f8a05a284723" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 17 05:33:35.937116 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:35.937084    2562 generic.go:356] "Generic (PLEG): container finished" podID="8a4b7bdf-dd83-41a6-bcd1-f8a05a284723" containerID="e560d5d7acb93e34ff1108b02905b697455493e751d771ae1b1901e5b70efbdb" exitCode=0
Jul 17 05:33:35.937546 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:35.937133    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh" event={"ID":"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723","Type":"ContainerDied","Data":"e560d5d7acb93e34ff1108b02905b697455493e751d771ae1b1901e5b70efbdb"}
Jul 17 05:33:35.937588 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:35.937556    2562 scope.go:117] "RemoveContainer" containerID="e560d5d7acb93e34ff1108b02905b697455493e751d771ae1b1901e5b70efbdb"
Jul 17 05:33:35.938587 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:35.938557    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:33:35.938662 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:35.938644    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:33:35.938713 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:35.938695    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:33:36.944061 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:36.944026    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh" event={"ID":"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723","Type":"ContainerStarted","Data":"93384ab855ef22f7097b9c7342ead4afd6b67d67cb7c801533dad1bc637d2f6a"}
Jul 17 05:33:37.525627 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:37.525595    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh"
Jul 17 05:33:37.525805 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:37.525772    2562 patch_prober.go:28] interesting pod/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 17 05:33:37.525850 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:37.525831    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh" podUID="8a4b7bdf-dd83-41a6-bcd1-f8a05a284723" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 17 05:33:38.526219 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:38.526178    2562 patch_prober.go:28] interesting pod/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 17 05:33:38.526716 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:38.526241    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh" podUID="8a4b7bdf-dd83-41a6-bcd1-f8a05a284723" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 17 05:33:39.526577 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:39.526540    2562 patch_prober.go:28] interesting pod/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 17 05:33:39.527010 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:39.526608    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh" podUID="8a4b7bdf-dd83-41a6-bcd1-f8a05a284723" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 17 05:33:40.525764 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:40.525727    2562 patch_prober.go:28] interesting pod/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 17 05:33:40.525945 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:40.525789    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh" podUID="8a4b7bdf-dd83-41a6-bcd1-f8a05a284723" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 17 05:33:41.529627 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:41.529593    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh"
Jul 17 05:33:41.530033 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:41.529872    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh"
Jul 17 05:33:41.530631 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:33:41.530613    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh"
Jul 17 05:34:18.081775 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:34:18.081740    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e73f00c0-9d866c87c-5rqhw"]
Jul 17 05:34:18.082328 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:34:18.081988    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e73f00c0-9d866c87c-5rqhw" podUID="e61eebe1-0c12-4b0b-8669-692bfe8a9e8c" containerName="maas-api" containerID="cri-o://871064e8860039d79319f33ed87ca13b59742e99ea93afa1dcd96c72425cd440" gracePeriod=30
Jul 17 05:34:18.336985 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:34:18.336910    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e73f00c0-9d866c87c-5rqhw"
Jul 17 05:34:18.404536 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:34:18.404505    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e61eebe1-0c12-4b0b-8669-692bfe8a9e8c-openshift-service-ca\") pod \"e61eebe1-0c12-4b0b-8669-692bfe8a9e8c\" (UID: \"e61eebe1-0c12-4b0b-8669-692bfe8a9e8c\") "
Jul 17 05:34:18.404710 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:34:18.404600    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e61eebe1-0c12-4b0b-8669-692bfe8a9e8c-maas-api-tls\") pod \"e61eebe1-0c12-4b0b-8669-692bfe8a9e8c\" (UID: \"e61eebe1-0c12-4b0b-8669-692bfe8a9e8c\") "
Jul 17 05:34:18.404710 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:34:18.404677    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p2bdw\" (UniqueName: \"kubernetes.io/projected/e61eebe1-0c12-4b0b-8669-692bfe8a9e8c-kube-api-access-p2bdw\") pod \"e61eebe1-0c12-4b0b-8669-692bfe8a9e8c\" (UID: \"e61eebe1-0c12-4b0b-8669-692bfe8a9e8c\") "
Jul 17 05:34:18.404833 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:34:18.404810    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e61eebe1-0c12-4b0b-8669-692bfe8a9e8c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e61eebe1-0c12-4b0b-8669-692bfe8a9e8c" (UID: "e61eebe1-0c12-4b0b-8669-692bfe8a9e8c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:34:18.404937 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:34:18.404921    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e61eebe1-0c12-4b0b-8669-692bfe8a9e8c-openshift-service-ca\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:34:18.406721 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:34:18.406693    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e61eebe1-0c12-4b0b-8669-692bfe8a9e8c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e61eebe1-0c12-4b0b-8669-692bfe8a9e8c" (UID: "e61eebe1-0c12-4b0b-8669-692bfe8a9e8c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 05:34:18.406927 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:34:18.406907    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e61eebe1-0c12-4b0b-8669-692bfe8a9e8c-kube-api-access-p2bdw" (OuterVolumeSpecName: "kube-api-access-p2bdw") pod "e61eebe1-0c12-4b0b-8669-692bfe8a9e8c" (UID: "e61eebe1-0c12-4b0b-8669-692bfe8a9e8c"). InnerVolumeSpecName "kube-api-access-p2bdw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:34:18.505674 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:34:18.505632    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e61eebe1-0c12-4b0b-8669-692bfe8a9e8c-maas-api-tls\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:34:18.505674 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:34:18.505671    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2bdw\" (UniqueName: \"kubernetes.io/projected/e61eebe1-0c12-4b0b-8669-692bfe8a9e8c-kube-api-access-p2bdw\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:34:19.173166 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:34:19.173072    2562 generic.go:356] "Generic (PLEG): container finished" podID="e61eebe1-0c12-4b0b-8669-692bfe8a9e8c" containerID="871064e8860039d79319f33ed87ca13b59742e99ea93afa1dcd96c72425cd440" exitCode=0
Jul 17 05:34:19.173166 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:34:19.173117    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e73f00c0-9d866c87c-5rqhw"
Jul 17 05:34:19.173166 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:34:19.173141    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e73f00c0-9d866c87c-5rqhw" event={"ID":"e61eebe1-0c12-4b0b-8669-692bfe8a9e8c","Type":"ContainerDied","Data":"871064e8860039d79319f33ed87ca13b59742e99ea93afa1dcd96c72425cd440"}
Jul 17 05:34:19.173710 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:34:19.173194    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e73f00c0-9d866c87c-5rqhw" event={"ID":"e61eebe1-0c12-4b0b-8669-692bfe8a9e8c","Type":"ContainerDied","Data":"9928fc684c683f3a8a81a743b73a9dc38141c79b7f5616bd36f0a177ba0e7f95"}
Jul 17 05:34:19.173710 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:34:19.173222    2562 scope.go:117] "RemoveContainer" containerID="871064e8860039d79319f33ed87ca13b59742e99ea93afa1dcd96c72425cd440"
Jul 17 05:34:19.182075 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:34:19.182021    2562 scope.go:117] "RemoveContainer" containerID="871064e8860039d79319f33ed87ca13b59742e99ea93afa1dcd96c72425cd440"
Jul 17 05:34:19.182308 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:34:19.182291    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"871064e8860039d79319f33ed87ca13b59742e99ea93afa1dcd96c72425cd440\": container with ID starting with 871064e8860039d79319f33ed87ca13b59742e99ea93afa1dcd96c72425cd440 not found: ID does not exist" containerID="871064e8860039d79319f33ed87ca13b59742e99ea93afa1dcd96c72425cd440"
Jul 17 05:34:19.182352 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:34:19.182317    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"871064e8860039d79319f33ed87ca13b59742e99ea93afa1dcd96c72425cd440"} err="failed to get container status \"871064e8860039d79319f33ed87ca13b59742e99ea93afa1dcd96c72425cd440\": rpc error: code = NotFound desc = could not find container \"871064e8860039d79319f33ed87ca13b59742e99ea93afa1dcd96c72425cd440\": container with ID starting with 871064e8860039d79319f33ed87ca13b59742e99ea93afa1dcd96c72425cd440 not found: ID does not exist"
Jul 17 05:34:19.191114 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:34:19.191088    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e73f00c0-9d866c87c-5rqhw"]
Jul 17 05:34:19.195275 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:34:19.195256    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e73f00c0-9d866c87c-5rqhw"]
Jul 17 05:34:20.884188 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:34:20.884150    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e61eebe1-0c12-4b0b-8669-692bfe8a9e8c" path="/var/lib/kubelet/pods/e61eebe1-0c12-4b0b-8669-692bfe8a9e8c/volumes"
Jul 17 05:35:12.992857 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:12.992781    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tx9gj_c1baeee2-350c-43c5-9a04-fb34378edb5c/ovn-acl-logging/0.log"
Jul 17 05:35:12.995406 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:12.995386    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tx9gj_c1baeee2-350c-43c5-9a04-fb34378edb5c/ovn-acl-logging/0.log"
Jul 17 05:35:15.013594 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:15.013564    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh"]
Jul 17 05:35:15.014089 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:15.013760    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh" podUID="8a4b7bdf-dd83-41a6-bcd1-f8a05a284723" containerName="istio-proxy" containerID="cri-o://93384ab855ef22f7097b9c7342ead4afd6b67d67cb7c801533dad1bc637d2f6a" gracePeriod=30
Jul 17 05:35:20.269481 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:20.269453    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh"
Jul 17 05:35:20.407451 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:20.407409    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-credential-socket\") pod \"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723\" (UID: \"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723\") "
Jul 17 05:35:20.407682 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:20.407466    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-workload-socket\") pod \"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723\" (UID: \"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723\") "
Jul 17 05:35:20.407682 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:20.407493    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-istio-envoy\") pod \"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723\" (UID: \"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723\") "
Jul 17 05:35:20.407682 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:20.407555    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6n4r7\" (UniqueName: \"kubernetes.io/projected/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-kube-api-access-6n4r7\") pod \"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723\" (UID: \"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723\") "
Jul 17 05:35:20.407682 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:20.407597    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-workload-certs\") pod \"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723\" (UID: \"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723\") "
Jul 17 05:35:20.407682 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:20.407630    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-istio-token\") pod \"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723\" (UID: \"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723\") "
Jul 17 05:35:20.407682 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:20.407662    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-istiod-ca-cert\") pod \"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723\" (UID: \"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723\") "
Jul 17 05:35:20.407997 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:20.407704    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-istio-podinfo\") pod \"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723\" (UID: \"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723\") "
Jul 17 05:35:20.407997 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:20.407734    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-istio-data\") pod \"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723\" (UID: \"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723\") "
Jul 17 05:35:20.407997 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:20.407757    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8a4b7bdf-dd83-41a6-bcd1-f8a05a284723" (UID: "8a4b7bdf-dd83-41a6-bcd1-f8a05a284723"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:35:20.407997 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:20.407853    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8a4b7bdf-dd83-41a6-bcd1-f8a05a284723" (UID: "8a4b7bdf-dd83-41a6-bcd1-f8a05a284723"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:35:20.407997 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:20.407868    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8a4b7bdf-dd83-41a6-bcd1-f8a05a284723" (UID: "8a4b7bdf-dd83-41a6-bcd1-f8a05a284723"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:35:20.408249 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:20.408151    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-credential-socket\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:35:20.408249 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:20.408167    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-workload-socket\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:35:20.408249 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:20.408176    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-workload-certs\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:35:20.408249 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:20.408212    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8a4b7bdf-dd83-41a6-bcd1-f8a05a284723" (UID: "8a4b7bdf-dd83-41a6-bcd1-f8a05a284723"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:35:20.409146 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:20.409117    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-istio-data" (OuterVolumeSpecName: "istio-data") pod "8a4b7bdf-dd83-41a6-bcd1-f8a05a284723" (UID: "8a4b7bdf-dd83-41a6-bcd1-f8a05a284723"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:35:20.410465 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:20.410437    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8a4b7bdf-dd83-41a6-bcd1-f8a05a284723" (UID: "8a4b7bdf-dd83-41a6-bcd1-f8a05a284723"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 05:35:20.410590 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:20.410572    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8a4b7bdf-dd83-41a6-bcd1-f8a05a284723" (UID: "8a4b7bdf-dd83-41a6-bcd1-f8a05a284723"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:35:20.410687 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:20.410667    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-kube-api-access-6n4r7" (OuterVolumeSpecName: "kube-api-access-6n4r7") pod "8a4b7bdf-dd83-41a6-bcd1-f8a05a284723" (UID: "8a4b7bdf-dd83-41a6-bcd1-f8a05a284723"). InnerVolumeSpecName "kube-api-access-6n4r7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:35:20.410733 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:20.410714    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-istio-token" (OuterVolumeSpecName: "istio-token") pod "8a4b7bdf-dd83-41a6-bcd1-f8a05a284723" (UID: "8a4b7bdf-dd83-41a6-bcd1-f8a05a284723"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:35:20.483263 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:20.483231    2562 generic.go:356] "Generic (PLEG): container finished" podID="8a4b7bdf-dd83-41a6-bcd1-f8a05a284723" containerID="93384ab855ef22f7097b9c7342ead4afd6b67d67cb7c801533dad1bc637d2f6a" exitCode=0
Jul 17 05:35:20.483447 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:20.483283    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh"
Jul 17 05:35:20.483447 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:20.483287    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh" event={"ID":"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723","Type":"ContainerDied","Data":"93384ab855ef22f7097b9c7342ead4afd6b67d67cb7c801533dad1bc637d2f6a"}
Jul 17 05:35:20.483447 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:20.483354    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh" event={"ID":"8a4b7bdf-dd83-41a6-bcd1-f8a05a284723","Type":"ContainerDied","Data":"431608bff2aad862b1261bbe9b13a0f85cc2c1822ed39a617fefb248916cc95e"}
Jul 17 05:35:20.483447 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:20.483377    2562 scope.go:117] "RemoveContainer" containerID="93384ab855ef22f7097b9c7342ead4afd6b67d67cb7c801533dad1bc637d2f6a"
Jul 17 05:35:20.492470 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:20.492451    2562 scope.go:117] "RemoveContainer" containerID="e560d5d7acb93e34ff1108b02905b697455493e751d771ae1b1901e5b70efbdb"
Jul 17 05:35:20.500300 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:20.500279    2562 scope.go:117] "RemoveContainer" containerID="93384ab855ef22f7097b9c7342ead4afd6b67d67cb7c801533dad1bc637d2f6a"
Jul 17 05:35:20.500554 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:35:20.500536    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93384ab855ef22f7097b9c7342ead4afd6b67d67cb7c801533dad1bc637d2f6a\": container with ID starting with 93384ab855ef22f7097b9c7342ead4afd6b67d67cb7c801533dad1bc637d2f6a not found: ID does not exist" containerID="93384ab855ef22f7097b9c7342ead4afd6b67d67cb7c801533dad1bc637d2f6a"
Jul 17 05:35:20.500623 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:20.500567    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93384ab855ef22f7097b9c7342ead4afd6b67d67cb7c801533dad1bc637d2f6a"} err="failed to get container status \"93384ab855ef22f7097b9c7342ead4afd6b67d67cb7c801533dad1bc637d2f6a\": rpc error: code = NotFound desc = could not find container \"93384ab855ef22f7097b9c7342ead4afd6b67d67cb7c801533dad1bc637d2f6a\": container with ID starting with 93384ab855ef22f7097b9c7342ead4afd6b67d67cb7c801533dad1bc637d2f6a not found: ID does not exist"
Jul 17 05:35:20.500623 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:20.500592    2562 scope.go:117] "RemoveContainer" containerID="e560d5d7acb93e34ff1108b02905b697455493e751d771ae1b1901e5b70efbdb"
Jul 17 05:35:20.500889 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:35:20.500867    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e560d5d7acb93e34ff1108b02905b697455493e751d771ae1b1901e5b70efbdb\": container with ID starting with e560d5d7acb93e34ff1108b02905b697455493e751d771ae1b1901e5b70efbdb not found: ID does not exist" containerID="e560d5d7acb93e34ff1108b02905b697455493e751d771ae1b1901e5b70efbdb"
Jul 17 05:35:20.501719 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:20.500893    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e560d5d7acb93e34ff1108b02905b697455493e751d771ae1b1901e5b70efbdb"} err="failed to get container status \"e560d5d7acb93e34ff1108b02905b697455493e751d771ae1b1901e5b70efbdb\": rpc error: code = NotFound desc = could not find container \"e560d5d7acb93e34ff1108b02905b697455493e751d771ae1b1901e5b70efbdb\": container with ID starting with e560d5d7acb93e34ff1108b02905b697455493e751d771ae1b1901e5b70efbdb not found: ID does not exist"
Jul 17 05:35:20.506068 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:20.506047    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh"]
Jul 17 05:35:20.508726 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:20.508706    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-istio-envoy\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:35:20.508726 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:20.508728    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6n4r7\" (UniqueName: \"kubernetes.io/projected/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-kube-api-access-6n4r7\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:35:20.508852 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:20.508737    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-istio-token\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:35:20.508852 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:20.508748    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-istiod-ca-cert\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:35:20.508852 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:20.508765    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-istio-podinfo\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:35:20.508852 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:20.508773    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723-istio-data\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:35:20.512113 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:20.512092    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-e73f00c0-openshift-default-6869488947-jgffh"]
Jul 17 05:35:20.884756 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:20.884720    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a4b7bdf-dd83-41a6-bcd1-f8a05a284723" path="/var/lib/kubelet/pods/8a4b7bdf-dd83-41a6-bcd1-f8a05a284723/volumes"
Jul 17 05:35:29.146226 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:29.146176    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-b2zbq"]
Jul 17 05:35:29.146775 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:29.146756    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a4dde0d-6c33-4847-a4a4-806a0da369aa" containerName="revoke-keys"
Jul 17 05:35:29.146864 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:29.146778    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a4dde0d-6c33-4847-a4a4-806a0da369aa" containerName="revoke-keys"
Jul 17 05:35:29.146864 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:29.146804    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a4b7bdf-dd83-41a6-bcd1-f8a05a284723" containerName="istio-proxy"
Jul 17 05:35:29.146864 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:29.146812    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a4b7bdf-dd83-41a6-bcd1-f8a05a284723" containerName="istio-proxy"
Jul 17 05:35:29.146864 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:29.146829    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a4b7bdf-dd83-41a6-bcd1-f8a05a284723" containerName="istio-proxy"
Jul 17 05:35:29.146864 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:29.146837    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a4b7bdf-dd83-41a6-bcd1-f8a05a284723" containerName="istio-proxy"
Jul 17 05:35:29.146864 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:29.146847    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e61eebe1-0c12-4b0b-8669-692bfe8a9e8c" containerName="maas-api"
Jul 17 05:35:29.146864 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:29.146855    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e61eebe1-0c12-4b0b-8669-692bfe8a9e8c" containerName="maas-api"
Jul 17 05:35:29.146864 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:29.146865    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a4dde0d-6c33-4847-a4a4-806a0da369aa" containerName="revoke-keys"
Jul 17 05:35:29.147297 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:29.146874    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a4dde0d-6c33-4847-a4a4-806a0da369aa" containerName="revoke-keys"
Jul 17 05:35:29.147297 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:29.146986    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a4dde0d-6c33-4847-a4a4-806a0da369aa" containerName="revoke-keys"
Jul 17 05:35:29.147297 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:29.147000    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a4b7bdf-dd83-41a6-bcd1-f8a05a284723" containerName="istio-proxy"
Jul 17 05:35:29.147297 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:29.147011    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e61eebe1-0c12-4b0b-8669-692bfe8a9e8c" containerName="maas-api"
Jul 17 05:35:29.147297 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:29.147022    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a4dde0d-6c33-4847-a4a4-806a0da369aa" containerName="revoke-keys"
Jul 17 05:35:29.147297 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:29.147031    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a4b7bdf-dd83-41a6-bcd1-f8a05a284723" containerName="istio-proxy"
Jul 17 05:35:29.150078 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:29.150053    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-b2zbq"
Jul 17 05:35:29.152678 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:29.152662    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qh8zj\""
Jul 17 05:35:29.156779 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:29.156754    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-b2zbq"]
Jul 17 05:35:29.184664 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:29.184634    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c39750da-771b-49ff-b6bd-21f50d769ae5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-c8a383ce-b2zbq\" (UID: \"c39750da-771b-49ff-b6bd-21f50d769ae5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-b2zbq"
Jul 17 05:35:29.285630 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:29.285587    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c39750da-771b-49ff-b6bd-21f50d769ae5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-c8a383ce-b2zbq\" (UID: \"c39750da-771b-49ff-b6bd-21f50d769ae5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-b2zbq"
Jul 17 05:35:29.286558 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:29.286527    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c39750da-771b-49ff-b6bd-21f50d769ae5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-c8a383ce-b2zbq\" (UID: \"c39750da-771b-49ff-b6bd-21f50d769ae5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-b2zbq"
Jul 17 05:35:29.462317 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:29.462229    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-b2zbq"
Jul 17 05:35:29.802586 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:29.802557    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-b2zbq"]
Jul 17 05:35:29.805021 ip-10-0-132-19 kubenswrapper[2562]: W0717 05:35:29.804995    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc39750da_771b_49ff_b6bd_21f50d769ae5.slice/crio-fe0b136d3f30b29dd040f978e19da90bebf1531109d59bfb6facc57711152ed6 WatchSource:0}: Error finding container fe0b136d3f30b29dd040f978e19da90bebf1531109d59bfb6facc57711152ed6: Status 404 returned error can't find the container with id fe0b136d3f30b29dd040f978e19da90bebf1531109d59bfb6facc57711152ed6
Jul 17 05:35:30.537115 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:30.537080    2562 generic.go:356] "Generic (PLEG): container finished" podID="c39750da-771b-49ff-b6bd-21f50d769ae5" containerID="4c2c894f6404d88b3cd9ecc4158f3a5391595ad67607432a902a51e080493a29" exitCode=6
Jul 17 05:35:30.537520 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:30.537168    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-b2zbq" event={"ID":"c39750da-771b-49ff-b6bd-21f50d769ae5","Type":"ContainerDied","Data":"4c2c894f6404d88b3cd9ecc4158f3a5391595ad67607432a902a51e080493a29"}
Jul 17 05:35:30.537520 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:30.537220    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-b2zbq" event={"ID":"c39750da-771b-49ff-b6bd-21f50d769ae5","Type":"ContainerStarted","Data":"fe0b136d3f30b29dd040f978e19da90bebf1531109d59bfb6facc57711152ed6"}
Jul 17 05:35:30.537520 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:30.537376    2562 scope.go:117] "RemoveContainer" containerID="4c2c894f6404d88b3cd9ecc4158f3a5391595ad67607432a902a51e080493a29"
Jul 17 05:35:31.545803 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:31.545708    2562 generic.go:356] "Generic (PLEG): container finished" podID="c39750da-771b-49ff-b6bd-21f50d769ae5" containerID="f51682ff52071c2db1c291874ebe3ae1bdc22842e11d5c09925fbea1ae2b31ac" exitCode=6
Jul 17 05:35:31.546288 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:31.545796    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-b2zbq" event={"ID":"c39750da-771b-49ff-b6bd-21f50d769ae5","Type":"ContainerDied","Data":"f51682ff52071c2db1c291874ebe3ae1bdc22842e11d5c09925fbea1ae2b31ac"}
Jul 17 05:35:31.546288 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:31.545860    2562 scope.go:117] "RemoveContainer" containerID="4c2c894f6404d88b3cd9ecc4158f3a5391595ad67607432a902a51e080493a29"
Jul 17 05:35:31.546288 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:31.546168    2562 scope.go:117] "RemoveContainer" containerID="f51682ff52071c2db1c291874ebe3ae1bdc22842e11d5c09925fbea1ae2b31ac"
Jul 17 05:35:31.546410 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:35:31.546371    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-c8a383ce-b2zbq_odh-ai-gateway-infra(c39750da-771b-49ff-b6bd-21f50d769ae5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-b2zbq" podUID="c39750da-771b-49ff-b6bd-21f50d769ae5"
Jul 17 05:35:32.553248 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:32.553217    2562 scope.go:117] "RemoveContainer" containerID="f51682ff52071c2db1c291874ebe3ae1bdc22842e11d5c09925fbea1ae2b31ac"
Jul 17 05:35:32.553644 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:35:32.553408    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-c8a383ce-b2zbq_odh-ai-gateway-infra(c39750da-771b-49ff-b6bd-21f50d769ae5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-b2zbq" podUID="c39750da-771b-49ff-b6bd-21f50d769ae5"
Jul 17 05:35:44.583930 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:44.583896    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-tzv4r"]
Jul 17 05:35:44.584394 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:44.584299    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a4dde0d-6c33-4847-a4a4-806a0da369aa" containerName="revoke-keys"
Jul 17 05:35:44.584394 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:44.584311    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a4dde0d-6c33-4847-a4a4-806a0da369aa" containerName="revoke-keys"
Jul 17 05:35:44.584394 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:44.584389    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a4dde0d-6c33-4847-a4a4-806a0da369aa" containerName="revoke-keys"
Jul 17 05:35:44.588422 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:44.588403    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-tzv4r"
Jul 17 05:35:44.594583 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:44.594561    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-tzv4r"]
Jul 17 05:35:44.740303 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:44.740262    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/14b08745-812a-4171-8866-6f782e1617c7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-bf0ad9db-tzv4r\" (UID: \"14b08745-812a-4171-8866-6f782e1617c7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-tzv4r"
Jul 17 05:35:44.841843 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:44.841756    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/14b08745-812a-4171-8866-6f782e1617c7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-bf0ad9db-tzv4r\" (UID: \"14b08745-812a-4171-8866-6f782e1617c7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-tzv4r"
Jul 17 05:35:44.842394 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:44.842374    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/14b08745-812a-4171-8866-6f782e1617c7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-bf0ad9db-tzv4r\" (UID: \"14b08745-812a-4171-8866-6f782e1617c7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-tzv4r"
Jul 17 05:35:44.899732 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:44.899701    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-tzv4r"
Jul 17 05:35:45.030326 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:45.030303    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-tzv4r"]
Jul 17 05:35:45.032571 ip-10-0-132-19 kubenswrapper[2562]: W0717 05:35:45.032533    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14b08745_812a_4171_8866_6f782e1617c7.slice/crio-025fbe30255462bcc624e612f89dd6619ba79db332a82f3497ea1c3123539189 WatchSource:0}: Error finding container 025fbe30255462bcc624e612f89dd6619ba79db332a82f3497ea1c3123539189: Status 404 returned error can't find the container with id 025fbe30255462bcc624e612f89dd6619ba79db332a82f3497ea1c3123539189
Jul 17 05:35:45.624826 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:45.624792    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-tzv4r" event={"ID":"14b08745-812a-4171-8866-6f782e1617c7","Type":"ContainerStarted","Data":"b41d7bc6437f25f341d1505660945a1c94b149b9b7d766ce7ac97f532e7c5456"}
Jul 17 05:35:45.625255 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:45.624879    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-tzv4r" event={"ID":"14b08745-812a-4171-8866-6f782e1617c7","Type":"ContainerStarted","Data":"025fbe30255462bcc624e612f89dd6619ba79db332a82f3497ea1c3123539189"}
Jul 17 05:35:45.641845 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:45.641797    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-tzv4r" podStartSLOduration=1.641782027 podStartE2EDuration="1.641782027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:35:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:35:45.640014444 +0000 UTC m=+2733.323608104" watchObservedRunningTime="2026-07-17 05:35:45.641782027 +0000 UTC m=+2733.325375685"
Jul 17 05:35:46.631745 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:46.631702    2562 generic.go:356] "Generic (PLEG): container finished" podID="14b08745-812a-4171-8866-6f782e1617c7" containerID="b41d7bc6437f25f341d1505660945a1c94b149b9b7d766ce7ac97f532e7c5456" exitCode=6
Jul 17 05:35:46.632182 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:46.631790    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-tzv4r" event={"ID":"14b08745-812a-4171-8866-6f782e1617c7","Type":"ContainerDied","Data":"b41d7bc6437f25f341d1505660945a1c94b149b9b7d766ce7ac97f532e7c5456"}
Jul 17 05:35:46.632182 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:46.632004    2562 scope.go:117] "RemoveContainer" containerID="b41d7bc6437f25f341d1505660945a1c94b149b9b7d766ce7ac97f532e7c5456"
Jul 17 05:35:47.639524 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:47.639490    2562 generic.go:356] "Generic (PLEG): container finished" podID="14b08745-812a-4171-8866-6f782e1617c7" containerID="d7ec29b9344bbf75719e92202907ebe8b94e33d125ed2b56c8299291bdeddbba" exitCode=6
Jul 17 05:35:47.639925 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:47.639554    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-tzv4r" event={"ID":"14b08745-812a-4171-8866-6f782e1617c7","Type":"ContainerDied","Data":"d7ec29b9344bbf75719e92202907ebe8b94e33d125ed2b56c8299291bdeddbba"}
Jul 17 05:35:47.639925 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:47.639620    2562 scope.go:117] "RemoveContainer" containerID="b41d7bc6437f25f341d1505660945a1c94b149b9b7d766ce7ac97f532e7c5456"
Jul 17 05:35:47.639925 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:47.639778    2562 scope.go:117] "RemoveContainer" containerID="d7ec29b9344bbf75719e92202907ebe8b94e33d125ed2b56c8299291bdeddbba"
Jul 17 05:35:47.640142 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:35:47.640067    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-bf0ad9db-tzv4r_odh-ai-gateway-infra(14b08745-812a-4171-8866-6f782e1617c7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-tzv4r" podUID="14b08745-812a-4171-8866-6f782e1617c7"
Jul 17 05:35:47.878821 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:47.878790    2562 scope.go:117] "RemoveContainer" containerID="f51682ff52071c2db1c291874ebe3ae1bdc22842e11d5c09925fbea1ae2b31ac"
Jul 17 05:35:48.646792 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:48.646756    2562 generic.go:356] "Generic (PLEG): container finished" podID="c39750da-771b-49ff-b6bd-21f50d769ae5" containerID="6d91112f524dea86a25d94cd06b21a1a8980487abce8cc1877b05b81c221b028" exitCode=6
Jul 17 05:35:48.647293 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:48.646834    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-b2zbq" event={"ID":"c39750da-771b-49ff-b6bd-21f50d769ae5","Type":"ContainerDied","Data":"6d91112f524dea86a25d94cd06b21a1a8980487abce8cc1877b05b81c221b028"}
Jul 17 05:35:48.647293 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:48.646897    2562 scope.go:117] "RemoveContainer" containerID="f51682ff52071c2db1c291874ebe3ae1bdc22842e11d5c09925fbea1ae2b31ac"
Jul 17 05:35:48.647293 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:48.647210    2562 scope.go:117] "RemoveContainer" containerID="6d91112f524dea86a25d94cd06b21a1a8980487abce8cc1877b05b81c221b028"
Jul 17 05:35:48.647505 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:35:48.647481    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-c8a383ce-b2zbq_odh-ai-gateway-infra(c39750da-771b-49ff-b6bd-21f50d769ae5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-b2zbq" podUID="c39750da-771b-49ff-b6bd-21f50d769ae5"
Jul 17 05:35:48.648687 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:48.648666    2562 scope.go:117] "RemoveContainer" containerID="d7ec29b9344bbf75719e92202907ebe8b94e33d125ed2b56c8299291bdeddbba"
Jul 17 05:35:48.648864 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:35:48.648847    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-bf0ad9db-tzv4r_odh-ai-gateway-infra(14b08745-812a-4171-8866-6f782e1617c7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-tzv4r" podUID="14b08745-812a-4171-8866-6f782e1617c7"
Jul 17 05:35:49.673533 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:49.673495    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-b2zbq"]
Jul 17 05:35:49.813761 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:49.813735    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-b2zbq"
Jul 17 05:35:49.894457 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:49.894421    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c39750da-771b-49ff-b6bd-21f50d769ae5-maas-api-service-ca\") pod \"c39750da-771b-49ff-b6bd-21f50d769ae5\" (UID: \"c39750da-771b-49ff-b6bd-21f50d769ae5\") "
Jul 17 05:35:49.894774 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:49.894752    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c39750da-771b-49ff-b6bd-21f50d769ae5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c39750da-771b-49ff-b6bd-21f50d769ae5" (UID: "c39750da-771b-49ff-b6bd-21f50d769ae5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:35:49.995839 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:49.995738    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c39750da-771b-49ff-b6bd-21f50d769ae5-maas-api-service-ca\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:35:50.662057 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:50.662023    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-b2zbq"
Jul 17 05:35:50.662057 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:50.662035    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-b2zbq" event={"ID":"c39750da-771b-49ff-b6bd-21f50d769ae5","Type":"ContainerDied","Data":"fe0b136d3f30b29dd040f978e19da90bebf1531109d59bfb6facc57711152ed6"}
Jul 17 05:35:50.662308 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:50.662095    2562 scope.go:117] "RemoveContainer" containerID="6d91112f524dea86a25d94cd06b21a1a8980487abce8cc1877b05b81c221b028"
Jul 17 05:35:50.686299 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:50.686267    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-b2zbq"]
Jul 17 05:35:50.694994 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:50.694151    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-b2zbq"]
Jul 17 05:35:50.884379 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:50.884343    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c39750da-771b-49ff-b6bd-21f50d769ae5" path="/var/lib/kubelet/pods/c39750da-771b-49ff-b6bd-21f50d769ae5/volumes"
Jul 17 05:35:51.038643 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:51.038558    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-ftrn9"]
Jul 17 05:35:51.038977 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:51.038947    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c39750da-771b-49ff-b6bd-21f50d769ae5" containerName="revoke-keys"
Jul 17 05:35:51.039017 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:51.038978    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c39750da-771b-49ff-b6bd-21f50d769ae5" containerName="revoke-keys"
Jul 17 05:35:51.039017 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:51.038989    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c39750da-771b-49ff-b6bd-21f50d769ae5" containerName="revoke-keys"
Jul 17 05:35:51.039017 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:51.038997    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c39750da-771b-49ff-b6bd-21f50d769ae5" containerName="revoke-keys"
Jul 17 05:35:51.039115 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:51.039079    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c39750da-771b-49ff-b6bd-21f50d769ae5" containerName="revoke-keys"
Jul 17 05:35:51.039115 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:51.039088    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c39750da-771b-49ff-b6bd-21f50d769ae5" containerName="revoke-keys"
Jul 17 05:35:51.043528 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:51.043511    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-ftrn9"
Jul 17 05:35:51.048923 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:51.048896    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-ftrn9"]
Jul 17 05:35:51.205930 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:51.205887    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4aa7343-dc75-4069-b421-b0df54215c76-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4e221674-ftrn9\" (UID: \"d4aa7343-dc75-4069-b421-b0df54215c76\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-ftrn9"
Jul 17 05:35:51.306734 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:51.306638    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4aa7343-dc75-4069-b421-b0df54215c76-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4e221674-ftrn9\" (UID: \"d4aa7343-dc75-4069-b421-b0df54215c76\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-ftrn9"
Jul 17 05:35:51.307314 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:51.307292    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4aa7343-dc75-4069-b421-b0df54215c76-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4e221674-ftrn9\" (UID: \"d4aa7343-dc75-4069-b421-b0df54215c76\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-ftrn9"
Jul 17 05:35:51.355618 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:51.355577    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-ftrn9"
Jul 17 05:35:51.490885 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:51.490860    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-ftrn9"]
Jul 17 05:35:51.492736 ip-10-0-132-19 kubenswrapper[2562]: W0717 05:35:51.492712    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4aa7343_dc75_4069_b421_b0df54215c76.slice/crio-2666356441fe6c056919ea1aab5903336f1d1f1aed69d79f0456fe0b66035b0c WatchSource:0}: Error finding container 2666356441fe6c056919ea1aab5903336f1d1f1aed69d79f0456fe0b66035b0c: Status 404 returned error can't find the container with id 2666356441fe6c056919ea1aab5903336f1d1f1aed69d79f0456fe0b66035b0c
Jul 17 05:35:51.668570 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:51.668534    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-ftrn9" event={"ID":"d4aa7343-dc75-4069-b421-b0df54215c76","Type":"ContainerStarted","Data":"2666356441fe6c056919ea1aab5903336f1d1f1aed69d79f0456fe0b66035b0c"}
Jul 17 05:35:52.675810 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:52.675780    2562 generic.go:356] "Generic (PLEG): container finished" podID="d4aa7343-dc75-4069-b421-b0df54215c76" containerID="e343c3c19a1a371ece29624059f44dfa81694c0f08832d5de173b855ccd34a1e" exitCode=6
Jul 17 05:35:52.676230 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:52.675832    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-ftrn9" event={"ID":"d4aa7343-dc75-4069-b421-b0df54215c76","Type":"ContainerDied","Data":"e343c3c19a1a371ece29624059f44dfa81694c0f08832d5de173b855ccd34a1e"}
Jul 17 05:35:52.676230 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:52.676086    2562 scope.go:117] "RemoveContainer" containerID="e343c3c19a1a371ece29624059f44dfa81694c0f08832d5de173b855ccd34a1e"
Jul 17 05:35:53.682831 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:53.682797    2562 generic.go:356] "Generic (PLEG): container finished" podID="d4aa7343-dc75-4069-b421-b0df54215c76" containerID="2d1c172a55bbf52a74b32e6f65599882b5c0e99c66d1ab2cf231ab4bd049986d" exitCode=6
Jul 17 05:35:53.683284 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:53.682880    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-ftrn9" event={"ID":"d4aa7343-dc75-4069-b421-b0df54215c76","Type":"ContainerDied","Data":"2d1c172a55bbf52a74b32e6f65599882b5c0e99c66d1ab2cf231ab4bd049986d"}
Jul 17 05:35:53.683284 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:53.682937    2562 scope.go:117] "RemoveContainer" containerID="e343c3c19a1a371ece29624059f44dfa81694c0f08832d5de173b855ccd34a1e"
Jul 17 05:35:53.683284 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:53.683148    2562 scope.go:117] "RemoveContainer" containerID="2d1c172a55bbf52a74b32e6f65599882b5c0e99c66d1ab2cf231ab4bd049986d"
Jul 17 05:35:53.683422 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:35:53.683366    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-4e221674-ftrn9_odh-ai-gateway-infra(d4aa7343-dc75-4069-b421-b0df54215c76)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-ftrn9" podUID="d4aa7343-dc75-4069-b421-b0df54215c76"
Jul 17 05:35:54.689787 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:54.689763    2562 scope.go:117] "RemoveContainer" containerID="2d1c172a55bbf52a74b32e6f65599882b5c0e99c66d1ab2cf231ab4bd049986d"
Jul 17 05:35:54.690230 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:35:54.690013    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-4e221674-ftrn9_odh-ai-gateway-infra(d4aa7343-dc75-4069-b421-b0df54215c76)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-ftrn9" podUID="d4aa7343-dc75-4069-b421-b0df54215c76"
Jul 17 05:35:59.878474 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:35:59.878435    2562 scope.go:117] "RemoveContainer" containerID="d7ec29b9344bbf75719e92202907ebe8b94e33d125ed2b56c8299291bdeddbba"
Jul 17 05:36:00.723240 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:00.723149    2562 generic.go:356] "Generic (PLEG): container finished" podID="14b08745-812a-4171-8866-6f782e1617c7" containerID="8c9992a8627c4d1b94864889b136fe46c92c8d81e7381bcf0976c1fce927e3ea" exitCode=6
Jul 17 05:36:00.723240 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:00.723222    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-tzv4r" event={"ID":"14b08745-812a-4171-8866-6f782e1617c7","Type":"ContainerDied","Data":"8c9992a8627c4d1b94864889b136fe46c92c8d81e7381bcf0976c1fce927e3ea"}
Jul 17 05:36:00.723429 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:00.723283    2562 scope.go:117] "RemoveContainer" containerID="d7ec29b9344bbf75719e92202907ebe8b94e33d125ed2b56c8299291bdeddbba"
Jul 17 05:36:00.723561 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:00.723543    2562 scope.go:117] "RemoveContainer" containerID="8c9992a8627c4d1b94864889b136fe46c92c8d81e7381bcf0976c1fce927e3ea"
Jul 17 05:36:00.723799 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:36:00.723778    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-bf0ad9db-tzv4r_odh-ai-gateway-infra(14b08745-812a-4171-8866-6f782e1617c7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-tzv4r" podUID="14b08745-812a-4171-8866-6f782e1617c7"
Jul 17 05:36:01.751581 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:01.751542    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-tzv4r"]
Jul 17 05:36:01.893226 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:01.893201    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-tzv4r"
Jul 17 05:36:01.915869 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:01.915842    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/14b08745-812a-4171-8866-6f782e1617c7-maas-api-service-ca\") pod \"14b08745-812a-4171-8866-6f782e1617c7\" (UID: \"14b08745-812a-4171-8866-6f782e1617c7\") "
Jul 17 05:36:01.916278 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:01.916249    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14b08745-812a-4171-8866-6f782e1617c7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "14b08745-812a-4171-8866-6f782e1617c7" (UID: "14b08745-812a-4171-8866-6f782e1617c7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:36:02.017124 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:02.017035    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/14b08745-812a-4171-8866-6f782e1617c7-maas-api-service-ca\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:36:02.743586 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:02.743550    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-tzv4r"
Jul 17 05:36:02.743586 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:02.743568    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-tzv4r" event={"ID":"14b08745-812a-4171-8866-6f782e1617c7","Type":"ContainerDied","Data":"025fbe30255462bcc624e612f89dd6619ba79db332a82f3497ea1c3123539189"}
Jul 17 05:36:02.743851 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:02.743633    2562 scope.go:117] "RemoveContainer" containerID="8c9992a8627c4d1b94864889b136fe46c92c8d81e7381bcf0976c1fce927e3ea"
Jul 17 05:36:02.769430 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:02.769393    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-tzv4r"]
Jul 17 05:36:02.771294 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:02.771272    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-tzv4r"]
Jul 17 05:36:02.884564 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:02.884527    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14b08745-812a-4171-8866-6f782e1617c7" path="/var/lib/kubelet/pods/14b08745-812a-4171-8866-6f782e1617c7/volumes"
Jul 17 05:36:07.239210 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:07.239167    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-xhbz4"]
Jul 17 05:36:07.239644 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:07.239576    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14b08745-812a-4171-8866-6f782e1617c7" containerName="revoke-keys"
Jul 17 05:36:07.239644 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:07.239588    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="14b08745-812a-4171-8866-6f782e1617c7" containerName="revoke-keys"
Jul 17 05:36:07.239644 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:07.239598    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14b08745-812a-4171-8866-6f782e1617c7" containerName="revoke-keys"
Jul 17 05:36:07.239644 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:07.239603    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="14b08745-812a-4171-8866-6f782e1617c7" containerName="revoke-keys"
Jul 17 05:36:07.239644 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:07.239612    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14b08745-812a-4171-8866-6f782e1617c7" containerName="revoke-keys"
Jul 17 05:36:07.239644 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:07.239618    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="14b08745-812a-4171-8866-6f782e1617c7" containerName="revoke-keys"
Jul 17 05:36:07.239644 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:07.239626    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c39750da-771b-49ff-b6bd-21f50d769ae5" containerName="revoke-keys"
Jul 17 05:36:07.239644 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:07.239631    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c39750da-771b-49ff-b6bd-21f50d769ae5" containerName="revoke-keys"
Jul 17 05:36:07.239895 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:07.239695    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="14b08745-812a-4171-8866-6f782e1617c7" containerName="revoke-keys"
Jul 17 05:36:07.239895 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:07.239701    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c39750da-771b-49ff-b6bd-21f50d769ae5" containerName="revoke-keys"
Jul 17 05:36:07.239895 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:07.239709    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="14b08745-812a-4171-8866-6f782e1617c7" containerName="revoke-keys"
Jul 17 05:36:07.243911 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:07.243891    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-xhbz4"
Jul 17 05:36:07.249810 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:07.249783    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-xhbz4"]
Jul 17 05:36:07.264623 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:07.264579    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5012f482-7044-464c-8309-7c473f1b5ded-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9eb8486e-xhbz4\" (UID: \"5012f482-7044-464c-8309-7c473f1b5ded\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-xhbz4"
Jul 17 05:36:07.365814 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:07.365775    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5012f482-7044-464c-8309-7c473f1b5ded-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9eb8486e-xhbz4\" (UID: \"5012f482-7044-464c-8309-7c473f1b5ded\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-xhbz4"
Jul 17 05:36:07.366464 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:07.366442    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5012f482-7044-464c-8309-7c473f1b5ded-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9eb8486e-xhbz4\" (UID: \"5012f482-7044-464c-8309-7c473f1b5ded\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-xhbz4"
Jul 17 05:36:07.556356 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:07.556259    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-xhbz4"
Jul 17 05:36:07.693997 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:07.693775    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-xhbz4"]
Jul 17 05:36:07.770240 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:07.770203    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-xhbz4" event={"ID":"5012f482-7044-464c-8309-7c473f1b5ded","Type":"ContainerStarted","Data":"99feab7046a94030ab68492abc3e943f05c0e2026b35314828ea98637cc458fb"}
Jul 17 05:36:08.776954 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:08.776861    2562 generic.go:356] "Generic (PLEG): container finished" podID="5012f482-7044-464c-8309-7c473f1b5ded" containerID="33110895c2dd9b6dbf79aff49753bd5efe8d34326b0bb6ca7cb191501bac03bf" exitCode=6
Jul 17 05:36:08.776954 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:08.776914    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-xhbz4" event={"ID":"5012f482-7044-464c-8309-7c473f1b5ded","Type":"ContainerDied","Data":"33110895c2dd9b6dbf79aff49753bd5efe8d34326b0bb6ca7cb191501bac03bf"}
Jul 17 05:36:08.777377 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:08.777143    2562 scope.go:117] "RemoveContainer" containerID="33110895c2dd9b6dbf79aff49753bd5efe8d34326b0bb6ca7cb191501bac03bf"
Jul 17 05:36:09.783508 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:09.783416    2562 generic.go:356] "Generic (PLEG): container finished" podID="5012f482-7044-464c-8309-7c473f1b5ded" containerID="4698c38e3d8c67bd8b8f096b37eed96feebe57fa0a8f0fcb726dfdedb1c78bba" exitCode=6
Jul 17 05:36:09.783947 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:09.783500    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-xhbz4" event={"ID":"5012f482-7044-464c-8309-7c473f1b5ded","Type":"ContainerDied","Data":"4698c38e3d8c67bd8b8f096b37eed96feebe57fa0a8f0fcb726dfdedb1c78bba"}
Jul 17 05:36:09.783947 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:09.783565    2562 scope.go:117] "RemoveContainer" containerID="33110895c2dd9b6dbf79aff49753bd5efe8d34326b0bb6ca7cb191501bac03bf"
Jul 17 05:36:09.783947 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:09.783832    2562 scope.go:117] "RemoveContainer" containerID="4698c38e3d8c67bd8b8f096b37eed96feebe57fa0a8f0fcb726dfdedb1c78bba"
Jul 17 05:36:09.784167 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:36:09.784071    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-9eb8486e-xhbz4_odh-ai-gateway-infra(5012f482-7044-464c-8309-7c473f1b5ded)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-xhbz4" podUID="5012f482-7044-464c-8309-7c473f1b5ded"
Jul 17 05:36:09.878454 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:09.878421    2562 scope.go:117] "RemoveContainer" containerID="2d1c172a55bbf52a74b32e6f65599882b5c0e99c66d1ab2cf231ab4bd049986d"
Jul 17 05:36:10.593246 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:10.593205    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6"]
Jul 17 05:36:10.594145 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:10.594119    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="14b08745-812a-4171-8866-6f782e1617c7" containerName="revoke-keys"
Jul 17 05:36:10.597176 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:10.597156    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6"
Jul 17 05:36:10.599892 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:10.599873    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-e1ab5d-openshift-default-dockercfg-4vbtv\""
Jul 17 05:36:10.609467 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:10.609443    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6"]
Jul 17 05:36:10.698315 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:10.698275    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77b797a3-8d68-4859-a401-2f3e1a830fc3-istio-token\") pod \"e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6\" (UID: \"77b797a3-8d68-4859-a401-2f3e1a830fc3\") " pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6"
Jul 17 05:36:10.698473 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:10.698324    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77b797a3-8d68-4859-a401-2f3e1a830fc3-credential-socket\") pod \"e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6\" (UID: \"77b797a3-8d68-4859-a401-2f3e1a830fc3\") " pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6"
Jul 17 05:36:10.698473 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:10.698371    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77b797a3-8d68-4859-a401-2f3e1a830fc3-istio-podinfo\") pod \"e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6\" (UID: \"77b797a3-8d68-4859-a401-2f3e1a830fc3\") " pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6"
Jul 17 05:36:10.698473 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:10.698393    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjddg\" (UniqueName: \"kubernetes.io/projected/77b797a3-8d68-4859-a401-2f3e1a830fc3-kube-api-access-rjddg\") pod \"e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6\" (UID: \"77b797a3-8d68-4859-a401-2f3e1a830fc3\") " pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6"
Jul 17 05:36:10.698473 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:10.698432    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77b797a3-8d68-4859-a401-2f3e1a830fc3-workload-certs\") pod \"e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6\" (UID: \"77b797a3-8d68-4859-a401-2f3e1a830fc3\") " pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6"
Jul 17 05:36:10.698693 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:10.698475    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77b797a3-8d68-4859-a401-2f3e1a830fc3-istio-envoy\") pod \"e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6\" (UID: \"77b797a3-8d68-4859-a401-2f3e1a830fc3\") " pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6"
Jul 17 05:36:10.698693 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:10.698526    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77b797a3-8d68-4859-a401-2f3e1a830fc3-istio-data\") pod \"e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6\" (UID: \"77b797a3-8d68-4859-a401-2f3e1a830fc3\") " pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6"
Jul 17 05:36:10.698693 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:10.698546    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77b797a3-8d68-4859-a401-2f3e1a830fc3-istiod-ca-cert\") pod \"e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6\" (UID: \"77b797a3-8d68-4859-a401-2f3e1a830fc3\") " pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6"
Jul 17 05:36:10.698693 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:10.698568    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77b797a3-8d68-4859-a401-2f3e1a830fc3-workload-socket\") pod \"e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6\" (UID: \"77b797a3-8d68-4859-a401-2f3e1a830fc3\") " pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6"
Jul 17 05:36:10.793259 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:10.793178    2562 scope.go:117] "RemoveContainer" containerID="4698c38e3d8c67bd8b8f096b37eed96feebe57fa0a8f0fcb726dfdedb1c78bba"
Jul 17 05:36:10.793656 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:36:10.793426    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-9eb8486e-xhbz4_odh-ai-gateway-infra(5012f482-7044-464c-8309-7c473f1b5ded)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-xhbz4" podUID="5012f482-7044-464c-8309-7c473f1b5ded"
Jul 17 05:36:10.795209 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:10.795049    2562 generic.go:356] "Generic (PLEG): container finished" podID="d4aa7343-dc75-4069-b421-b0df54215c76" containerID="c73caa0f844fcd1a32c2d2a5cb29f4720a24036c1a3f27913cfe15fc8b23f8de" exitCode=6
Jul 17 05:36:10.795209 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:10.795126    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-ftrn9" event={"ID":"d4aa7343-dc75-4069-b421-b0df54215c76","Type":"ContainerDied","Data":"c73caa0f844fcd1a32c2d2a5cb29f4720a24036c1a3f27913cfe15fc8b23f8de"}
Jul 17 05:36:10.795209 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:10.795194    2562 scope.go:117] "RemoveContainer" containerID="2d1c172a55bbf52a74b32e6f65599882b5c0e99c66d1ab2cf231ab4bd049986d"
Jul 17 05:36:10.795663 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:10.795566    2562 scope.go:117] "RemoveContainer" containerID="c73caa0f844fcd1a32c2d2a5cb29f4720a24036c1a3f27913cfe15fc8b23f8de"
Jul 17 05:36:10.795800 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:36:10.795770    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-4e221674-ftrn9_odh-ai-gateway-infra(d4aa7343-dc75-4069-b421-b0df54215c76)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-ftrn9" podUID="d4aa7343-dc75-4069-b421-b0df54215c76"
Jul 17 05:36:10.800285 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:10.800199    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77b797a3-8d68-4859-a401-2f3e1a830fc3-istio-data\") pod \"e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6\" (UID: \"77b797a3-8d68-4859-a401-2f3e1a830fc3\") " pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6"
Jul 17 05:36:10.800285 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:10.800236    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77b797a3-8d68-4859-a401-2f3e1a830fc3-istiod-ca-cert\") pod \"e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6\" (UID: \"77b797a3-8d68-4859-a401-2f3e1a830fc3\") " pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6"
Jul 17 05:36:10.800285 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:10.800270    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77b797a3-8d68-4859-a401-2f3e1a830fc3-workload-socket\") pod \"e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6\" (UID: \"77b797a3-8d68-4859-a401-2f3e1a830fc3\") " pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6"
Jul 17 05:36:10.800567 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:10.800324    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77b797a3-8d68-4859-a401-2f3e1a830fc3-istio-token\") pod \"e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6\" (UID: \"77b797a3-8d68-4859-a401-2f3e1a830fc3\") " pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6"
Jul 17 05:36:10.800567 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:10.800366    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77b797a3-8d68-4859-a401-2f3e1a830fc3-credential-socket\") pod \"e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6\" (UID: \"77b797a3-8d68-4859-a401-2f3e1a830fc3\") " pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6"
Jul 17 05:36:10.800567 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:10.800419    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77b797a3-8d68-4859-a401-2f3e1a830fc3-istio-podinfo\") pod \"e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6\" (UID: \"77b797a3-8d68-4859-a401-2f3e1a830fc3\") " pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6"
Jul 17 05:36:10.800567 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:10.800449    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjddg\" (UniqueName: \"kubernetes.io/projected/77b797a3-8d68-4859-a401-2f3e1a830fc3-kube-api-access-rjddg\") pod \"e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6\" (UID: \"77b797a3-8d68-4859-a401-2f3e1a830fc3\") " pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6"
Jul 17 05:36:10.800567 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:10.800493    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77b797a3-8d68-4859-a401-2f3e1a830fc3-workload-certs\") pod \"e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6\" (UID: \"77b797a3-8d68-4859-a401-2f3e1a830fc3\") " pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6"
Jul 17 05:36:10.800567 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:10.800525    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77b797a3-8d68-4859-a401-2f3e1a830fc3-istio-envoy\") pod \"e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6\" (UID: \"77b797a3-8d68-4859-a401-2f3e1a830fc3\") " pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6"
Jul 17 05:36:10.800876 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:10.800705    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77b797a3-8d68-4859-a401-2f3e1a830fc3-istio-data\") pod \"e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6\" (UID: \"77b797a3-8d68-4859-a401-2f3e1a830fc3\") " pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6"
Jul 17 05:36:10.801403 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:10.801042    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77b797a3-8d68-4859-a401-2f3e1a830fc3-istiod-ca-cert\") pod \"e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6\" (UID: \"77b797a3-8d68-4859-a401-2f3e1a830fc3\") " pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6"
Jul 17 05:36:10.801403 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:10.801207    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77b797a3-8d68-4859-a401-2f3e1a830fc3-credential-socket\") pod \"e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6\" (UID: \"77b797a3-8d68-4859-a401-2f3e1a830fc3\") " pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6"
Jul 17 05:36:10.801403 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:10.801223    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77b797a3-8d68-4859-a401-2f3e1a830fc3-workload-certs\") pod \"e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6\" (UID: \"77b797a3-8d68-4859-a401-2f3e1a830fc3\") " pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6"
Jul 17 05:36:10.801403 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:10.801274    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77b797a3-8d68-4859-a401-2f3e1a830fc3-workload-socket\") pod \"e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6\" (UID: \"77b797a3-8d68-4859-a401-2f3e1a830fc3\") " pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6"
Jul 17 05:36:10.802881 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:10.802858    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77b797a3-8d68-4859-a401-2f3e1a830fc3-istio-envoy\") pod \"e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6\" (UID: \"77b797a3-8d68-4859-a401-2f3e1a830fc3\") " pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6"
Jul 17 05:36:10.803448 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:10.803431    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77b797a3-8d68-4859-a401-2f3e1a830fc3-istio-podinfo\") pod \"e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6\" (UID: \"77b797a3-8d68-4859-a401-2f3e1a830fc3\") " pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6"
Jul 17 05:36:10.825058 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:10.825030    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjddg\" (UniqueName: \"kubernetes.io/projected/77b797a3-8d68-4859-a401-2f3e1a830fc3-kube-api-access-rjddg\") pod \"e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6\" (UID: \"77b797a3-8d68-4859-a401-2f3e1a830fc3\") " pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6"
Jul 17 05:36:10.828923 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:10.828900    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77b797a3-8d68-4859-a401-2f3e1a830fc3-istio-token\") pod \"e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6\" (UID: \"77b797a3-8d68-4859-a401-2f3e1a830fc3\") " pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6"
Jul 17 05:36:10.908138 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:10.908106    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6"
Jul 17 05:36:11.050698 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:11.050524    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6"]
Jul 17 05:36:11.053081 ip-10-0-132-19 kubenswrapper[2562]: W0717 05:36:11.053043    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77b797a3_8d68_4859_a401_2f3e1a830fc3.slice/crio-943e58b489ede0478e5ed6391315422b4f58885e4c8542c29a90c8180cf72473 WatchSource:0}: Error finding container 943e58b489ede0478e5ed6391315422b4f58885e4c8542c29a90c8180cf72473: Status 404 returned error can't find the container with id 943e58b489ede0478e5ed6391315422b4f58885e4c8542c29a90c8180cf72473
Jul 17 05:36:11.055464 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:11.055424    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:36:11.055554 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:11.055510    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:36:11.055592 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:11.055557    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:36:11.803144 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:11.803105    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6" event={"ID":"77b797a3-8d68-4859-a401-2f3e1a830fc3","Type":"ContainerStarted","Data":"7bcb067e4d1a54718f3b94b4d47a400216c9b00a2b5e856d7ea2aea830d60995"}
Jul 17 05:36:11.803548 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:11.803166    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6" event={"ID":"77b797a3-8d68-4859-a401-2f3e1a830fc3","Type":"ContainerStarted","Data":"943e58b489ede0478e5ed6391315422b4f58885e4c8542c29a90c8180cf72473"}
Jul 17 05:36:11.830178 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:11.830118    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6" podStartSLOduration=1.830099917 podStartE2EDuration="1.830099917s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:36:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:36:11.828304567 +0000 UTC m=+2759.511898226" watchObservedRunningTime="2026-07-17 05:36:11.830099917 +0000 UTC m=+2759.513693576"
Jul 17 05:36:11.859018 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:11.858980    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-ftrn9"]
Jul 17 05:36:11.908938 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:11.908909    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6"
Jul 17 05:36:11.998682 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:11.998654    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-ftrn9"
Jul 17 05:36:12.112820 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:12.112785    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4aa7343-dc75-4069-b421-b0df54215c76-maas-api-service-ca\") pod \"d4aa7343-dc75-4069-b421-b0df54215c76\" (UID: \"d4aa7343-dc75-4069-b421-b0df54215c76\") "
Jul 17 05:36:12.113593 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:12.113541    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4aa7343-dc75-4069-b421-b0df54215c76-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d4aa7343-dc75-4069-b421-b0df54215c76" (UID: "d4aa7343-dc75-4069-b421-b0df54215c76"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:36:12.115055 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:12.114014    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4aa7343-dc75-4069-b421-b0df54215c76-maas-api-service-ca\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:36:12.812705 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:12.812672    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-ftrn9"
Jul 17 05:36:12.813208 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:12.812697    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-ftrn9" event={"ID":"d4aa7343-dc75-4069-b421-b0df54215c76","Type":"ContainerDied","Data":"2666356441fe6c056919ea1aab5903336f1d1f1aed69d79f0456fe0b66035b0c"}
Jul 17 05:36:12.813208 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:12.812802    2562 scope.go:117] "RemoveContainer" containerID="c73caa0f844fcd1a32c2d2a5cb29f4720a24036c1a3f27913cfe15fc8b23f8de"
Jul 17 05:36:12.834305 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:12.834274    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-ftrn9"]
Jul 17 05:36:12.838029 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:12.838002    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-ftrn9"]
Jul 17 05:36:12.884277 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:12.884248    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4aa7343-dc75-4069-b421-b0df54215c76" path="/var/lib/kubelet/pods/d4aa7343-dc75-4069-b421-b0df54215c76/volumes"
Jul 17 05:36:12.914025 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:12.913990    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6"
Jul 17 05:36:13.818610 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:13.818576    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6"
Jul 17 05:36:13.819619 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:13.819602    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6"
Jul 17 05:36:13.878129 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:13.878091    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e1ab5d-bf6959f49-shnbz"]
Jul 17 05:36:13.878720 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:13.878701    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4aa7343-dc75-4069-b421-b0df54215c76" containerName="revoke-keys"
Jul 17 05:36:13.878720 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:13.878722    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4aa7343-dc75-4069-b421-b0df54215c76" containerName="revoke-keys"
Jul 17 05:36:13.878893 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:13.878737    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4aa7343-dc75-4069-b421-b0df54215c76" containerName="revoke-keys"
Jul 17 05:36:13.878893 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:13.878747    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4aa7343-dc75-4069-b421-b0df54215c76" containerName="revoke-keys"
Jul 17 05:36:13.878893 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:13.878788    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4aa7343-dc75-4069-b421-b0df54215c76" containerName="revoke-keys"
Jul 17 05:36:13.878893 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:13.878797    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4aa7343-dc75-4069-b421-b0df54215c76" containerName="revoke-keys"
Jul 17 05:36:13.878893 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:13.878889    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4aa7343-dc75-4069-b421-b0df54215c76" containerName="revoke-keys"
Jul 17 05:36:13.879156 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:13.878899    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4aa7343-dc75-4069-b421-b0df54215c76" containerName="revoke-keys"
Jul 17 05:36:13.891166 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:13.891137    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e1ab5d-bf6959f49-shnbz"
Jul 17 05:36:13.894591 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:13.894560    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e1ab5d-bf6959f49-shnbz"]
Jul 17 05:36:13.894775 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:13.894749    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-xwcd7\""
Jul 17 05:36:13.894874 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:13.894590    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-e1ab5d-serving-cert\""
Jul 17 05:36:13.930093 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:13.930053    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxcq5\" (UniqueName: \"kubernetes.io/projected/0124abf8-492d-4fcf-85e5-e9659fe3be44-kube-api-access-rxcq5\") pod \"maas-api-e2e-inf-a-e1ab5d-bf6959f49-shnbz\" (UID: \"0124abf8-492d-4fcf-85e5-e9659fe3be44\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e1ab5d-bf6959f49-shnbz"
Jul 17 05:36:13.930290 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:13.930239    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0124abf8-492d-4fcf-85e5-e9659fe3be44-maas-api-tls\") pod \"maas-api-e2e-inf-a-e1ab5d-bf6959f49-shnbz\" (UID: \"0124abf8-492d-4fcf-85e5-e9659fe3be44\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e1ab5d-bf6959f49-shnbz"
Jul 17 05:36:13.930290 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:13.930278    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0124abf8-492d-4fcf-85e5-e9659fe3be44-openshift-service-ca\") pod \"maas-api-e2e-inf-a-e1ab5d-bf6959f49-shnbz\" (UID: \"0124abf8-492d-4fcf-85e5-e9659fe3be44\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e1ab5d-bf6959f49-shnbz"
Jul 17 05:36:14.031420 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:14.031387    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxcq5\" (UniqueName: \"kubernetes.io/projected/0124abf8-492d-4fcf-85e5-e9659fe3be44-kube-api-access-rxcq5\") pod \"maas-api-e2e-inf-a-e1ab5d-bf6959f49-shnbz\" (UID: \"0124abf8-492d-4fcf-85e5-e9659fe3be44\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e1ab5d-bf6959f49-shnbz"
Jul 17 05:36:14.031606 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:14.031534    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0124abf8-492d-4fcf-85e5-e9659fe3be44-maas-api-tls\") pod \"maas-api-e2e-inf-a-e1ab5d-bf6959f49-shnbz\" (UID: \"0124abf8-492d-4fcf-85e5-e9659fe3be44\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e1ab5d-bf6959f49-shnbz"
Jul 17 05:36:14.031606 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:14.031570    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0124abf8-492d-4fcf-85e5-e9659fe3be44-openshift-service-ca\") pod \"maas-api-e2e-inf-a-e1ab5d-bf6959f49-shnbz\" (UID: \"0124abf8-492d-4fcf-85e5-e9659fe3be44\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e1ab5d-bf6959f49-shnbz"
Jul 17 05:36:14.031699 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:36:14.031678    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-e1ab5d-serving-cert: secret "maas-api-e2e-inf-a-e1ab5d-serving-cert" not found
Jul 17 05:36:14.031762 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:36:14.031752    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0124abf8-492d-4fcf-85e5-e9659fe3be44-maas-api-tls podName:0124abf8-492d-4fcf-85e5-e9659fe3be44 nodeName:}" failed. No retries permitted until 2026-07-17 05:36:14.531729155 +0000 UTC m=+2762.215322790 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0124abf8-492d-4fcf-85e5-e9659fe3be44-maas-api-tls") pod "maas-api-e2e-inf-a-e1ab5d-bf6959f49-shnbz" (UID: "0124abf8-492d-4fcf-85e5-e9659fe3be44") : secret "maas-api-e2e-inf-a-e1ab5d-serving-cert" not found
Jul 17 05:36:14.032237 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:14.032220    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0124abf8-492d-4fcf-85e5-e9659fe3be44-openshift-service-ca\") pod \"maas-api-e2e-inf-a-e1ab5d-bf6959f49-shnbz\" (UID: \"0124abf8-492d-4fcf-85e5-e9659fe3be44\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e1ab5d-bf6959f49-shnbz"
Jul 17 05:36:14.040955 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:14.040923    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxcq5\" (UniqueName: \"kubernetes.io/projected/0124abf8-492d-4fcf-85e5-e9659fe3be44-kube-api-access-rxcq5\") pod \"maas-api-e2e-inf-a-e1ab5d-bf6959f49-shnbz\" (UID: \"0124abf8-492d-4fcf-85e5-e9659fe3be44\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e1ab5d-bf6959f49-shnbz"
Jul 17 05:36:14.537507 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:14.537470    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0124abf8-492d-4fcf-85e5-e9659fe3be44-maas-api-tls\") pod \"maas-api-e2e-inf-a-e1ab5d-bf6959f49-shnbz\" (UID: \"0124abf8-492d-4fcf-85e5-e9659fe3be44\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e1ab5d-bf6959f49-shnbz"
Jul 17 05:36:14.540177 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:14.540154    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0124abf8-492d-4fcf-85e5-e9659fe3be44-maas-api-tls\") pod \"maas-api-e2e-inf-a-e1ab5d-bf6959f49-shnbz\" (UID: \"0124abf8-492d-4fcf-85e5-e9659fe3be44\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e1ab5d-bf6959f49-shnbz"
Jul 17 05:36:14.806281 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:14.806196    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e1ab5d-bf6959f49-shnbz"
Jul 17 05:36:14.981364 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:14.981335    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e1ab5d-bf6959f49-shnbz"]
Jul 17 05:36:14.982998 ip-10-0-132-19 kubenswrapper[2562]: W0717 05:36:14.982955    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0124abf8_492d_4fcf_85e5_e9659fe3be44.slice/crio-b35f99058dc687333f64e06d8b243ef6a792518c10de8cd92718c04574e7da27 WatchSource:0}: Error finding container b35f99058dc687333f64e06d8b243ef6a792518c10de8cd92718c04574e7da27: Status 404 returned error can't find the container with id b35f99058dc687333f64e06d8b243ef6a792518c10de8cd92718c04574e7da27
Jul 17 05:36:15.831045 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:15.830999    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e1ab5d-bf6959f49-shnbz" event={"ID":"0124abf8-492d-4fcf-85e5-e9659fe3be44","Type":"ContainerStarted","Data":"d0663531664af4a1b8ae430f884acf04309f6019c3a8746db033831e63eb015b"}
Jul 17 05:36:15.831338 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:15.831070    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e1ab5d-bf6959f49-shnbz" event={"ID":"0124abf8-492d-4fcf-85e5-e9659fe3be44","Type":"ContainerStarted","Data":"b35f99058dc687333f64e06d8b243ef6a792518c10de8cd92718c04574e7da27"}
Jul 17 05:36:15.831338 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:15.831286    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e1ab5d-bf6959f49-shnbz"
Jul 17 05:36:15.848206 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:15.848148    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e1ab5d-bf6959f49-shnbz" podStartSLOduration=2.37864382 podStartE2EDuration="2.8481261s" totalImagesPullingTime="469.48228ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:36:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:36:15.848096969 +0000 UTC m=+2763.531690628" watchObservedRunningTime="2026-07-17 05:36:15.8481261 +0000 UTC m=+2763.531719760"
Jul 17 05:36:19.005764 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:19.003765    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd"]
Jul 17 05:36:19.005764 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:19.004544    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4aa7343-dc75-4069-b421-b0df54215c76" containerName="revoke-keys"
Jul 17 05:36:19.008321 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:19.008298    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd"
Jul 17 05:36:19.012359 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:19.012332    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-63fb52-openshift-default-dockercfg-hxr6b\""
Jul 17 05:36:19.022094 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:19.022062    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd"]
Jul 17 05:36:19.091922 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:19.091888    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-istio-data\") pod \"e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd\" (UID: \"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e\") " pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd"
Jul 17 05:36:19.092150 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:19.092049    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-istio-envoy\") pod \"e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd\" (UID: \"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e\") " pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd"
Jul 17 05:36:19.092150 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:19.092117    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-workload-certs\") pod \"e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd\" (UID: \"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e\") " pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd"
Jul 17 05:36:19.092288 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:19.092160    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-istio-token\") pod \"e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd\" (UID: \"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e\") " pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd"
Jul 17 05:36:19.092288 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:19.092249    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-credential-socket\") pod \"e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd\" (UID: \"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e\") " pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd"
Jul 17 05:36:19.092288 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:19.092281    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-istiod-ca-cert\") pod \"e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd\" (UID: \"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e\") " pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd"
Jul 17 05:36:19.092452 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:19.092328    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84pb5\" (UniqueName: \"kubernetes.io/projected/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-kube-api-access-84pb5\") pod \"e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd\" (UID: \"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e\") " pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd"
Jul 17 05:36:19.092452 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:19.092369    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-istio-podinfo\") pod \"e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd\" (UID: \"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e\") " pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd"
Jul 17 05:36:19.092452 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:19.092443    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-workload-socket\") pod \"e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd\" (UID: \"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e\") " pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd"
Jul 17 05:36:19.193509 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:19.193471    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-workload-certs\") pod \"e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd\" (UID: \"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e\") " pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd"
Jul 17 05:36:19.193509 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:19.193509    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-istio-token\") pod \"e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd\" (UID: \"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e\") " pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd"
Jul 17 05:36:19.193776 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:19.193570    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-credential-socket\") pod \"e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd\" (UID: \"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e\") " pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd"
Jul 17 05:36:19.193776 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:19.193596    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-istiod-ca-cert\") pod \"e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd\" (UID: \"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e\") " pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd"
Jul 17 05:36:19.193776 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:19.193621    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84pb5\" (UniqueName: \"kubernetes.io/projected/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-kube-api-access-84pb5\") pod \"e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd\" (UID: \"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e\") " pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd"
Jul 17 05:36:19.193776 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:19.193654    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-istio-podinfo\") pod \"e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd\" (UID: \"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e\") " pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd"
Jul 17 05:36:19.193776 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:19.193703    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-workload-socket\") pod \"e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd\" (UID: \"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e\") " pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd"
Jul 17 05:36:19.193776 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:19.193766    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-istio-data\") pod \"e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd\" (UID: \"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e\") " pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd"
Jul 17 05:36:19.194128 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:19.193839    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-istio-envoy\") pod \"e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd\" (UID: \"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e\") " pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd"
Jul 17 05:36:19.194128 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:19.193946    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-workload-certs\") pod \"e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd\" (UID: \"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e\") " pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd"
Jul 17 05:36:19.194128 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:19.194073    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-credential-socket\") pod \"e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd\" (UID: \"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e\") " pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd"
Jul 17 05:36:19.194246 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:19.194124    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-workload-socket\") pod \"e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd\" (UID: \"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e\") " pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd"
Jul 17 05:36:19.194326 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:19.194307    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-istio-data\") pod \"e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd\" (UID: \"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e\") " pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd"
Jul 17 05:36:19.194452 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:19.194432    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-istiod-ca-cert\") pod \"e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd\" (UID: \"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e\") " pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd"
Jul 17 05:36:19.196224 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:19.196195    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-istio-envoy\") pod \"e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd\" (UID: \"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e\") " pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd"
Jul 17 05:36:19.196507 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:19.196486    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-istio-podinfo\") pod \"e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd\" (UID: \"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e\") " pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd"
Jul 17 05:36:19.203147 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:19.203124    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-istio-token\") pod \"e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd\" (UID: \"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e\") " pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd"
Jul 17 05:36:19.203428 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:19.203405    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84pb5\" (UniqueName: \"kubernetes.io/projected/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-kube-api-access-84pb5\") pod \"e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd\" (UID: \"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e\") " pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd"
Jul 17 05:36:19.322755 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:19.322717    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd"
Jul 17 05:36:19.510113 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:19.510084    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd"]
Jul 17 05:36:19.511684 ip-10-0-132-19 kubenswrapper[2562]: W0717 05:36:19.511661    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b56dd72_cddf_484d_8b14_b77e3a2d1c9e.slice/crio-306861ee318491d32d1b78e6a5e84b0245c2a81ea90544fe258bb141d3f975f7 WatchSource:0}: Error finding container 306861ee318491d32d1b78e6a5e84b0245c2a81ea90544fe258bb141d3f975f7: Status 404 returned error can't find the container with id 306861ee318491d32d1b78e6a5e84b0245c2a81ea90544fe258bb141d3f975f7
Jul 17 05:36:19.514090 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:19.514051    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:36:19.514173 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:19.514125    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:36:19.514173 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:19.514152    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:36:19.853515 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:19.853405    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd" event={"ID":"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e","Type":"ContainerStarted","Data":"f16b8abd219c07a375503f9178c56f487da48aa16a21f1dcd557a334fcc20100"}
Jul 17 05:36:19.853515 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:19.853463    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd" event={"ID":"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e","Type":"ContainerStarted","Data":"306861ee318491d32d1b78e6a5e84b0245c2a81ea90544fe258bb141d3f975f7"}
Jul 17 05:36:19.877854 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:19.877797    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd" podStartSLOduration=1.877780137 podStartE2EDuration="1.877780137s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:36:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:36:19.875419765 +0000 UTC m=+2767.559013440" watchObservedRunningTime="2026-07-17 05:36:19.877780137 +0000 UTC m=+2767.561373874"
Jul 17 05:36:20.323395 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:20.323357    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd"
Jul 17 05:36:20.328523 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:20.328493    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd"
Jul 17 05:36:20.858913 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:20.858885    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd"
Jul 17 05:36:20.860038 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:20.860017    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd"
Jul 17 05:36:21.842760 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:21.842732    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e1ab5d-bf6959f49-shnbz"
Jul 17 05:36:21.949487 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:21.949449    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-63fb52-6f5cc68758-pbmph"]
Jul 17 05:36:21.953479 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:21.953453    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-63fb52-6f5cc68758-pbmph"
Jul 17 05:36:21.956185 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:21.956162    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-63fb52-serving-cert\""
Jul 17 05:36:21.965260 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:21.965236    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-63fb52-6f5cc68758-pbmph"]
Jul 17 05:36:22.026251 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:22.026214    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d013611-bf77-4fb2-bd72-e1458cf4c090-openshift-service-ca\") pod \"maas-api-e2e-inf-b-63fb52-6f5cc68758-pbmph\" (UID: \"4d013611-bf77-4fb2-bd72-e1458cf4c090\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-63fb52-6f5cc68758-pbmph"
Jul 17 05:36:22.026467 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:22.026294    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8kgz\" (UniqueName: \"kubernetes.io/projected/4d013611-bf77-4fb2-bd72-e1458cf4c090-kube-api-access-l8kgz\") pod \"maas-api-e2e-inf-b-63fb52-6f5cc68758-pbmph\" (UID: \"4d013611-bf77-4fb2-bd72-e1458cf4c090\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-63fb52-6f5cc68758-pbmph"
Jul 17 05:36:22.026467 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:22.026412    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d013611-bf77-4fb2-bd72-e1458cf4c090-maas-api-tls\") pod \"maas-api-e2e-inf-b-63fb52-6f5cc68758-pbmph\" (UID: \"4d013611-bf77-4fb2-bd72-e1458cf4c090\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-63fb52-6f5cc68758-pbmph"
Jul 17 05:36:22.127680 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:22.127591    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d013611-bf77-4fb2-bd72-e1458cf4c090-maas-api-tls\") pod \"maas-api-e2e-inf-b-63fb52-6f5cc68758-pbmph\" (UID: \"4d013611-bf77-4fb2-bd72-e1458cf4c090\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-63fb52-6f5cc68758-pbmph"
Jul 17 05:36:22.127680 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:22.127657    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d013611-bf77-4fb2-bd72-e1458cf4c090-openshift-service-ca\") pod \"maas-api-e2e-inf-b-63fb52-6f5cc68758-pbmph\" (UID: \"4d013611-bf77-4fb2-bd72-e1458cf4c090\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-63fb52-6f5cc68758-pbmph"
Jul 17 05:36:22.127904 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:22.127712    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8kgz\" (UniqueName: \"kubernetes.io/projected/4d013611-bf77-4fb2-bd72-e1458cf4c090-kube-api-access-l8kgz\") pod \"maas-api-e2e-inf-b-63fb52-6f5cc68758-pbmph\" (UID: \"4d013611-bf77-4fb2-bd72-e1458cf4c090\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-63fb52-6f5cc68758-pbmph"
Jul 17 05:36:22.127904 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:36:22.127741    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-63fb52-serving-cert: secret "maas-api-e2e-inf-b-63fb52-serving-cert" not found
Jul 17 05:36:22.127904 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:36:22.127803    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d013611-bf77-4fb2-bd72-e1458cf4c090-maas-api-tls podName:4d013611-bf77-4fb2-bd72-e1458cf4c090 nodeName:}" failed. No retries permitted until 2026-07-17 05:36:22.627787145 +0000 UTC m=+2770.311380779 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4d013611-bf77-4fb2-bd72-e1458cf4c090-maas-api-tls") pod "maas-api-e2e-inf-b-63fb52-6f5cc68758-pbmph" (UID: "4d013611-bf77-4fb2-bd72-e1458cf4c090") : secret "maas-api-e2e-inf-b-63fb52-serving-cert" not found
Jul 17 05:36:22.128382 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:22.128362    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d013611-bf77-4fb2-bd72-e1458cf4c090-openshift-service-ca\") pod \"maas-api-e2e-inf-b-63fb52-6f5cc68758-pbmph\" (UID: \"4d013611-bf77-4fb2-bd72-e1458cf4c090\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-63fb52-6f5cc68758-pbmph"
Jul 17 05:36:22.139427 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:22.139402    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8kgz\" (UniqueName: \"kubernetes.io/projected/4d013611-bf77-4fb2-bd72-e1458cf4c090-kube-api-access-l8kgz\") pod \"maas-api-e2e-inf-b-63fb52-6f5cc68758-pbmph\" (UID: \"4d013611-bf77-4fb2-bd72-e1458cf4c090\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-63fb52-6f5cc68758-pbmph"
Jul 17 05:36:22.634223 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:22.634177    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d013611-bf77-4fb2-bd72-e1458cf4c090-maas-api-tls\") pod \"maas-api-e2e-inf-b-63fb52-6f5cc68758-pbmph\" (UID: \"4d013611-bf77-4fb2-bd72-e1458cf4c090\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-63fb52-6f5cc68758-pbmph"
Jul 17 05:36:22.636803 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:22.636778    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d013611-bf77-4fb2-bd72-e1458cf4c090-maas-api-tls\") pod \"maas-api-e2e-inf-b-63fb52-6f5cc68758-pbmph\" (UID: \"4d013611-bf77-4fb2-bd72-e1458cf4c090\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-63fb52-6f5cc68758-pbmph"
Jul 17 05:36:22.866463 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:22.866429    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-63fb52-6f5cc68758-pbmph"
Jul 17 05:36:23.022163 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:23.022138    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-63fb52-6f5cc68758-pbmph"]
Jul 17 05:36:23.024307 ip-10-0-132-19 kubenswrapper[2562]: W0717 05:36:23.024268    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d013611_bf77_4fb2_bd72_e1458cf4c090.slice/crio-829bade22a3784cbfd8f8c7e86ddd7cd466a018bd54ca81d13cb02447154bf90 WatchSource:0}: Error finding container 829bade22a3784cbfd8f8c7e86ddd7cd466a018bd54ca81d13cb02447154bf90: Status 404 returned error can't find the container with id 829bade22a3784cbfd8f8c7e86ddd7cd466a018bd54ca81d13cb02447154bf90
Jul 17 05:36:23.880520 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:23.880477    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-63fb52-6f5cc68758-pbmph" event={"ID":"4d013611-bf77-4fb2-bd72-e1458cf4c090","Type":"ContainerStarted","Data":"e0b0fef0b2a79662a082113116f8f83505b60b364fcbba2260ccd1a2370bada3"}
Jul 17 05:36:23.881087 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:23.880545    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-63fb52-6f5cc68758-pbmph" event={"ID":"4d013611-bf77-4fb2-bd72-e1458cf4c090","Type":"ContainerStarted","Data":"829bade22a3784cbfd8f8c7e86ddd7cd466a018bd54ca81d13cb02447154bf90"}
Jul 17 05:36:23.881087 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:23.880668    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-63fb52-6f5cc68758-pbmph"
Jul 17 05:36:23.898542 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:23.898483    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-63fb52-6f5cc68758-pbmph" podStartSLOduration=2.595701477 podStartE2EDuration="2.898465178s" totalImagesPullingTime="302.763701ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:36:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:36:23.896543898 +0000 UTC m=+2771.580137581" watchObservedRunningTime="2026-07-17 05:36:23.898465178 +0000 UTC m=+2771.582058834"
Jul 17 05:36:24.885895 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:24.885869    2562 scope.go:117] "RemoveContainer" containerID="4698c38e3d8c67bd8b8f096b37eed96feebe57fa0a8f0fcb726dfdedb1c78bba"
Jul 17 05:36:25.892624 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:25.892592    2562 generic.go:356] "Generic (PLEG): container finished" podID="5012f482-7044-464c-8309-7c473f1b5ded" containerID="dfaa4a7789d2509df5c577e6a790f95396132b8e7415f0e4cdecf6b074062744" exitCode=6
Jul 17 05:36:25.893093 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:25.892664    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-xhbz4" event={"ID":"5012f482-7044-464c-8309-7c473f1b5ded","Type":"ContainerDied","Data":"dfaa4a7789d2509df5c577e6a790f95396132b8e7415f0e4cdecf6b074062744"}
Jul 17 05:36:25.893093 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:25.892725    2562 scope.go:117] "RemoveContainer" containerID="4698c38e3d8c67bd8b8f096b37eed96feebe57fa0a8f0fcb726dfdedb1c78bba"
Jul 17 05:36:25.893093 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:25.893035    2562 scope.go:117] "RemoveContainer" containerID="dfaa4a7789d2509df5c577e6a790f95396132b8e7415f0e4cdecf6b074062744"
Jul 17 05:36:25.893283 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:36:25.893270    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-9eb8486e-xhbz4_odh-ai-gateway-infra(5012f482-7044-464c-8309-7c473f1b5ded)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-xhbz4" podUID="5012f482-7044-464c-8309-7c473f1b5ded"
Jul 17 05:36:26.919022 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:26.918987    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-xhbz4"]
Jul 17 05:36:27.101872 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:27.101849    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-xhbz4"
Jul 17 05:36:27.188734 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:27.188639    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5012f482-7044-464c-8309-7c473f1b5ded-maas-api-service-ca\") pod \"5012f482-7044-464c-8309-7c473f1b5ded\" (UID: \"5012f482-7044-464c-8309-7c473f1b5ded\") "
Jul 17 05:36:27.189131 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:27.189108    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5012f482-7044-464c-8309-7c473f1b5ded-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5012f482-7044-464c-8309-7c473f1b5ded" (UID: "5012f482-7044-464c-8309-7c473f1b5ded"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:36:27.289898 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:27.289847    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5012f482-7044-464c-8309-7c473f1b5ded-maas-api-service-ca\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:36:27.909510 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:27.909469    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-xhbz4" event={"ID":"5012f482-7044-464c-8309-7c473f1b5ded","Type":"ContainerDied","Data":"99feab7046a94030ab68492abc3e943f05c0e2026b35314828ea98637cc458fb"}
Jul 17 05:36:27.909510 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:27.909501    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-xhbz4"
Jul 17 05:36:27.909772 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:27.909538    2562 scope.go:117] "RemoveContainer" containerID="dfaa4a7789d2509df5c577e6a790f95396132b8e7415f0e4cdecf6b074062744"
Jul 17 05:36:27.933107 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:27.933072    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-xhbz4"]
Jul 17 05:36:27.940944 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:27.940920    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-xhbz4"]
Jul 17 05:36:28.885186 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:28.885144    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5012f482-7044-464c-8309-7c473f1b5ded" path="/var/lib/kubelet/pods/5012f482-7044-464c-8309-7c473f1b5ded/volumes"
Jul 17 05:36:29.860237 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:29.860187    2562 patch_prober.go:28] interesting pod/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 17 05:36:29.860724 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:29.860284    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6" podUID="77b797a3-8d68-4859-a401-2f3e1a830fc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 17 05:36:29.892405 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:29.892376    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-63fb52-6f5cc68758-pbmph"
Jul 17 05:36:30.930420 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:30.930384    2562 generic.go:356] "Generic (PLEG): container finished" podID="77b797a3-8d68-4859-a401-2f3e1a830fc3" containerID="7bcb067e4d1a54718f3b94b4d47a400216c9b00a2b5e856d7ea2aea830d60995" exitCode=0
Jul 17 05:36:30.930875 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:30.930456    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6" event={"ID":"77b797a3-8d68-4859-a401-2f3e1a830fc3","Type":"ContainerDied","Data":"7bcb067e4d1a54718f3b94b4d47a400216c9b00a2b5e856d7ea2aea830d60995"}
Jul 17 05:36:30.930875 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:30.930846    2562 scope.go:117] "RemoveContainer" containerID="7bcb067e4d1a54718f3b94b4d47a400216c9b00a2b5e856d7ea2aea830d60995"
Jul 17 05:36:30.931855 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:30.931823    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:36:30.931953 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:30.931891    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:36:30.931953 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:30.931923    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:36:31.937328 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:31.937291    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6" event={"ID":"77b797a3-8d68-4859-a401-2f3e1a830fc3","Type":"ContainerStarted","Data":"32a698461bee0a6447b520bb4ac5d668416539840edd774520151da5332b36ab"}
Jul 17 05:36:32.909321 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:32.909286    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6"
Jul 17 05:36:32.909538 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:32.909407    2562 patch_prober.go:28] interesting pod/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Jul 17 05:36:32.909538 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:32.909449    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6" podUID="77b797a3-8d68-4859-a401-2f3e1a830fc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Jul 17 05:36:33.909355 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:33.909310    2562 patch_prober.go:28] interesting pod/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Jul 17 05:36:33.909750 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:33.909377    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6" podUID="77b797a3-8d68-4859-a401-2f3e1a830fc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Jul 17 05:36:34.908555 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:34.908517    2562 patch_prober.go:28] interesting pod/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Jul 17 05:36:34.908745 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:34.908582    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6" podUID="77b797a3-8d68-4859-a401-2f3e1a830fc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Jul 17 05:36:35.908631 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:35.908595    2562 patch_prober.go:28] interesting pod/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Jul 17 05:36:35.909134 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:35.908658    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6" podUID="77b797a3-8d68-4859-a401-2f3e1a830fc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Jul 17 05:36:36.901166 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:36.901127    2562 patch_prober.go:28] interesting pod/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 17 05:36:36.901364 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:36.901216    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd" podUID="1b56dd72-cddf-484d-8b14-b77e3a2d1c9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 17 05:36:36.912195 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:36.912165    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6"
Jul 17 05:36:36.912628 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:36.912577    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6"
Jul 17 05:36:36.913431 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:36.913406    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6"
Jul 17 05:36:37.975989 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:37.975931    2562 generic.go:356] "Generic (PLEG): container finished" podID="1b56dd72-cddf-484d-8b14-b77e3a2d1c9e" containerID="f16b8abd219c07a375503f9178c56f487da48aa16a21f1dcd557a334fcc20100" exitCode=0
Jul 17 05:36:37.976446 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:37.976006    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd" event={"ID":"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e","Type":"ContainerDied","Data":"f16b8abd219c07a375503f9178c56f487da48aa16a21f1dcd557a334fcc20100"}
Jul 17 05:36:37.976590 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:37.976574    2562 scope.go:117] "RemoveContainer" containerID="f16b8abd219c07a375503f9178c56f487da48aa16a21f1dcd557a334fcc20100"
Jul 17 05:36:37.977563 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:37.977537    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:36:37.977636 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:37.977604    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:36:37.977673 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:37.977638    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:36:38.983390 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:38.983348    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd" event={"ID":"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e","Type":"ContainerStarted","Data":"88b031b2c3f0ae8308e1e1f3da8b29ce2691156a5d6a4f9f91f0445af6b6f0f6"}
Jul 17 05:36:39.323077 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:39.323041    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd"
Jul 17 05:36:39.323360 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:39.323336    2562 patch_prober.go:28] interesting pod/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Jul 17 05:36:39.323458 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:39.323394    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd" podUID="1b56dd72-cddf-484d-8b14-b77e3a2d1c9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Jul 17 05:36:40.323751 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:40.323713    2562 patch_prober.go:28] interesting pod/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Jul 17 05:36:40.324186 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:40.323772    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd" podUID="1b56dd72-cddf-484d-8b14-b77e3a2d1c9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Jul 17 05:36:41.323194 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:41.323155    2562 patch_prober.go:28] interesting pod/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Jul 17 05:36:41.323386 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:41.323230    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd" podUID="1b56dd72-cddf-484d-8b14-b77e3a2d1c9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Jul 17 05:36:42.323523 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:42.323489    2562 patch_prober.go:28] interesting pod/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Jul 17 05:36:42.323938 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:42.323545    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd" podUID="1b56dd72-cddf-484d-8b14-b77e3a2d1c9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Jul 17 05:36:43.327088 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:43.327057    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd"
Jul 17 05:36:43.327549 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:43.327340    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd"
Jul 17 05:36:43.328093 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:36:43.328074    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd"
Jul 17 05:37:36.674783 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:36.674738    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e1ab5d-bf6959f49-shnbz"]
Jul 17 05:37:36.675285 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:36.675016    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e1ab5d-bf6959f49-shnbz" podUID="0124abf8-492d-4fcf-85e5-e9659fe3be44" containerName="maas-api" containerID="cri-o://d0663531664af4a1b8ae430f884acf04309f6019c3a8746db033831e63eb015b" gracePeriod=30
Jul 17 05:37:36.836675 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:36.836638    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e1ab5d-bf6959f49-shnbz" podUID="0124abf8-492d-4fcf-85e5-e9659fe3be44" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.63:8443/health\": dial tcp 10.134.0.63:8443: connect: connection refused"
Jul 17 05:37:36.933734 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:36.933674    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e1ab5d-bf6959f49-shnbz"
Jul 17 05:37:37.047941 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:37.047904    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rxcq5\" (UniqueName: \"kubernetes.io/projected/0124abf8-492d-4fcf-85e5-e9659fe3be44-kube-api-access-rxcq5\") pod \"0124abf8-492d-4fcf-85e5-e9659fe3be44\" (UID: \"0124abf8-492d-4fcf-85e5-e9659fe3be44\") "
Jul 17 05:37:37.047941 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:37.047942    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0124abf8-492d-4fcf-85e5-e9659fe3be44-openshift-service-ca\") pod \"0124abf8-492d-4fcf-85e5-e9659fe3be44\" (UID: \"0124abf8-492d-4fcf-85e5-e9659fe3be44\") "
Jul 17 05:37:37.048183 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:37.048128    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0124abf8-492d-4fcf-85e5-e9659fe3be44-maas-api-tls\") pod \"0124abf8-492d-4fcf-85e5-e9659fe3be44\" (UID: \"0124abf8-492d-4fcf-85e5-e9659fe3be44\") "
Jul 17 05:37:37.048441 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:37.048405    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0124abf8-492d-4fcf-85e5-e9659fe3be44-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0124abf8-492d-4fcf-85e5-e9659fe3be44" (UID: "0124abf8-492d-4fcf-85e5-e9659fe3be44"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:37:37.050325 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:37.050301    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0124abf8-492d-4fcf-85e5-e9659fe3be44-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0124abf8-492d-4fcf-85e5-e9659fe3be44" (UID: "0124abf8-492d-4fcf-85e5-e9659fe3be44"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 05:37:37.050434 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:37.050413    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0124abf8-492d-4fcf-85e5-e9659fe3be44-kube-api-access-rxcq5" (OuterVolumeSpecName: "kube-api-access-rxcq5") pod "0124abf8-492d-4fcf-85e5-e9659fe3be44" (UID: "0124abf8-492d-4fcf-85e5-e9659fe3be44"). InnerVolumeSpecName "kube-api-access-rxcq5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:37:37.149486 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:37.149444    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rxcq5\" (UniqueName: \"kubernetes.io/projected/0124abf8-492d-4fcf-85e5-e9659fe3be44-kube-api-access-rxcq5\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:37:37.149486 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:37.149479    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0124abf8-492d-4fcf-85e5-e9659fe3be44-openshift-service-ca\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:37:37.149486 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:37.149490    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0124abf8-492d-4fcf-85e5-e9659fe3be44-maas-api-tls\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:37:37.289799 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:37.289702    2562 generic.go:356] "Generic (PLEG): container finished" podID="0124abf8-492d-4fcf-85e5-e9659fe3be44" containerID="d0663531664af4a1b8ae430f884acf04309f6019c3a8746db033831e63eb015b" exitCode=0
Jul 17 05:37:37.289799 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:37.289759    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e1ab5d-bf6959f49-shnbz"
Jul 17 05:37:37.289799 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:37.289790    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e1ab5d-bf6959f49-shnbz" event={"ID":"0124abf8-492d-4fcf-85e5-e9659fe3be44","Type":"ContainerDied","Data":"d0663531664af4a1b8ae430f884acf04309f6019c3a8746db033831e63eb015b"}
Jul 17 05:37:37.290136 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:37.289843    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e1ab5d-bf6959f49-shnbz" event={"ID":"0124abf8-492d-4fcf-85e5-e9659fe3be44","Type":"ContainerDied","Data":"b35f99058dc687333f64e06d8b243ef6a792518c10de8cd92718c04574e7da27"}
Jul 17 05:37:37.290136 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:37.289864    2562 scope.go:117] "RemoveContainer" containerID="d0663531664af4a1b8ae430f884acf04309f6019c3a8746db033831e63eb015b"
Jul 17 05:37:37.299707 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:37.299684    2562 scope.go:117] "RemoveContainer" containerID="d0663531664af4a1b8ae430f884acf04309f6019c3a8746db033831e63eb015b"
Jul 17 05:37:37.300029 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:37:37.300008    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0663531664af4a1b8ae430f884acf04309f6019c3a8746db033831e63eb015b\": container with ID starting with d0663531664af4a1b8ae430f884acf04309f6019c3a8746db033831e63eb015b not found: ID does not exist" containerID="d0663531664af4a1b8ae430f884acf04309f6019c3a8746db033831e63eb015b"
Jul 17 05:37:37.300093 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:37.300039    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0663531664af4a1b8ae430f884acf04309f6019c3a8746db033831e63eb015b"} err="failed to get container status \"d0663531664af4a1b8ae430f884acf04309f6019c3a8746db033831e63eb015b\": rpc error: code = NotFound desc = could not find container \"d0663531664af4a1b8ae430f884acf04309f6019c3a8746db033831e63eb015b\": container with ID starting with d0663531664af4a1b8ae430f884acf04309f6019c3a8746db033831e63eb015b not found: ID does not exist"
Jul 17 05:37:37.312368 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:37.312340    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e1ab5d-bf6959f49-shnbz"]
Jul 17 05:37:37.319286 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:37.319256    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e1ab5d-bf6959f49-shnbz"]
Jul 17 05:37:38.883787 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:38.883753    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0124abf8-492d-4fcf-85e5-e9659fe3be44" path="/var/lib/kubelet/pods/0124abf8-492d-4fcf-85e5-e9659fe3be44/volumes"
Jul 17 05:37:51.859560 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:51.859529    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6"]
Jul 17 05:37:51.860130 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:51.859741    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6" podUID="77b797a3-8d68-4859-a401-2f3e1a830fc3" containerName="istio-proxy" containerID="cri-o://32a698461bee0a6447b520bb4ac5d668416539840edd774520151da5332b36ab" gracePeriod=30
Jul 17 05:37:51.914878 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:51.914841    2562 patch_prober.go:28] interesting pod/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 17 05:37:51.915066 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:51.914907    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6" podUID="77b797a3-8d68-4859-a401-2f3e1a830fc3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 17 05:37:57.115136 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:57.115112    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6"
Jul 17 05:37:57.247663 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:57.247582    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77b797a3-8d68-4859-a401-2f3e1a830fc3-istio-envoy\") pod \"77b797a3-8d68-4859-a401-2f3e1a830fc3\" (UID: \"77b797a3-8d68-4859-a401-2f3e1a830fc3\") "
Jul 17 05:37:57.247663 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:57.247620    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rjddg\" (UniqueName: \"kubernetes.io/projected/77b797a3-8d68-4859-a401-2f3e1a830fc3-kube-api-access-rjddg\") pod \"77b797a3-8d68-4859-a401-2f3e1a830fc3\" (UID: \"77b797a3-8d68-4859-a401-2f3e1a830fc3\") "
Jul 17 05:37:57.247663 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:57.247646    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77b797a3-8d68-4859-a401-2f3e1a830fc3-workload-socket\") pod \"77b797a3-8d68-4859-a401-2f3e1a830fc3\" (UID: \"77b797a3-8d68-4859-a401-2f3e1a830fc3\") "
Jul 17 05:37:57.247956 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:57.247687    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77b797a3-8d68-4859-a401-2f3e1a830fc3-credential-socket\") pod \"77b797a3-8d68-4859-a401-2f3e1a830fc3\" (UID: \"77b797a3-8d68-4859-a401-2f3e1a830fc3\") "
Jul 17 05:37:57.247956 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:57.247756    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77b797a3-8d68-4859-a401-2f3e1a830fc3-istio-podinfo\") pod \"77b797a3-8d68-4859-a401-2f3e1a830fc3\" (UID: \"77b797a3-8d68-4859-a401-2f3e1a830fc3\") "
Jul 17 05:37:57.247956 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:57.247770    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77b797a3-8d68-4859-a401-2f3e1a830fc3-workload-certs\") pod \"77b797a3-8d68-4859-a401-2f3e1a830fc3\" (UID: \"77b797a3-8d68-4859-a401-2f3e1a830fc3\") "
Jul 17 05:37:57.247956 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:57.247793    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77b797a3-8d68-4859-a401-2f3e1a830fc3-istio-data\") pod \"77b797a3-8d68-4859-a401-2f3e1a830fc3\" (UID: \"77b797a3-8d68-4859-a401-2f3e1a830fc3\") "
Jul 17 05:37:57.247956 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:57.247838    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77b797a3-8d68-4859-a401-2f3e1a830fc3-istiod-ca-cert\") pod \"77b797a3-8d68-4859-a401-2f3e1a830fc3\" (UID: \"77b797a3-8d68-4859-a401-2f3e1a830fc3\") "
Jul 17 05:37:57.247956 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:57.247889    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77b797a3-8d68-4859-a401-2f3e1a830fc3-istio-token\") pod \"77b797a3-8d68-4859-a401-2f3e1a830fc3\" (UID: \"77b797a3-8d68-4859-a401-2f3e1a830fc3\") "
Jul 17 05:37:57.248303 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:57.248003    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77b797a3-8d68-4859-a401-2f3e1a830fc3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "77b797a3-8d68-4859-a401-2f3e1a830fc3" (UID: "77b797a3-8d68-4859-a401-2f3e1a830fc3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:37:57.248303 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:57.248030    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77b797a3-8d68-4859-a401-2f3e1a830fc3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "77b797a3-8d68-4859-a401-2f3e1a830fc3" (UID: "77b797a3-8d68-4859-a401-2f3e1a830fc3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:37:57.248581 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:57.248313    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77b797a3-8d68-4859-a401-2f3e1a830fc3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "77b797a3-8d68-4859-a401-2f3e1a830fc3" (UID: "77b797a3-8d68-4859-a401-2f3e1a830fc3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:37:57.248581 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:57.248326    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77b797a3-8d68-4859-a401-2f3e1a830fc3-workload-socket\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:37:57.248581 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:57.248364    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77b797a3-8d68-4859-a401-2f3e1a830fc3-credential-socket\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:37:57.248854 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:57.248825    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77b797a3-8d68-4859-a401-2f3e1a830fc3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "77b797a3-8d68-4859-a401-2f3e1a830fc3" (UID: "77b797a3-8d68-4859-a401-2f3e1a830fc3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:37:57.248954 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:57.248927    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77b797a3-8d68-4859-a401-2f3e1a830fc3-istio-data" (OuterVolumeSpecName: "istio-data") pod "77b797a3-8d68-4859-a401-2f3e1a830fc3" (UID: "77b797a3-8d68-4859-a401-2f3e1a830fc3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:37:57.250137 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:57.250104    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77b797a3-8d68-4859-a401-2f3e1a830fc3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "77b797a3-8d68-4859-a401-2f3e1a830fc3" (UID: "77b797a3-8d68-4859-a401-2f3e1a830fc3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:37:57.250249 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:57.250191    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/77b797a3-8d68-4859-a401-2f3e1a830fc3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "77b797a3-8d68-4859-a401-2f3e1a830fc3" (UID: "77b797a3-8d68-4859-a401-2f3e1a830fc3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 05:37:57.250249 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:57.250202    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77b797a3-8d68-4859-a401-2f3e1a830fc3-kube-api-access-rjddg" (OuterVolumeSpecName: "kube-api-access-rjddg") pod "77b797a3-8d68-4859-a401-2f3e1a830fc3" (UID: "77b797a3-8d68-4859-a401-2f3e1a830fc3"). InnerVolumeSpecName "kube-api-access-rjddg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:37:57.250249 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:57.250217    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77b797a3-8d68-4859-a401-2f3e1a830fc3-istio-token" (OuterVolumeSpecName: "istio-token") pod "77b797a3-8d68-4859-a401-2f3e1a830fc3" (UID: "77b797a3-8d68-4859-a401-2f3e1a830fc3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:37:57.348862 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:57.348834    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77b797a3-8d68-4859-a401-2f3e1a830fc3-istio-envoy\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:37:57.348862 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:57.348859    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjddg\" (UniqueName: \"kubernetes.io/projected/77b797a3-8d68-4859-a401-2f3e1a830fc3-kube-api-access-rjddg\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:37:57.349043 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:57.348870    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77b797a3-8d68-4859-a401-2f3e1a830fc3-istio-podinfo\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:37:57.349043 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:57.348881    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77b797a3-8d68-4859-a401-2f3e1a830fc3-workload-certs\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:37:57.349043 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:57.348889    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77b797a3-8d68-4859-a401-2f3e1a830fc3-istio-data\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:37:57.349043 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:57.348896    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77b797a3-8d68-4859-a401-2f3e1a830fc3-istiod-ca-cert\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:37:57.349043 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:57.348904    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77b797a3-8d68-4859-a401-2f3e1a830fc3-istio-token\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:37:57.398392 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:57.398355    2562 generic.go:356] "Generic (PLEG): container finished" podID="77b797a3-8d68-4859-a401-2f3e1a830fc3" containerID="32a698461bee0a6447b520bb4ac5d668416539840edd774520151da5332b36ab" exitCode=0
Jul 17 05:37:57.398533 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:57.398412    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6"
Jul 17 05:37:57.398533 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:57.398418    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6" event={"ID":"77b797a3-8d68-4859-a401-2f3e1a830fc3","Type":"ContainerDied","Data":"32a698461bee0a6447b520bb4ac5d668416539840edd774520151da5332b36ab"}
Jul 17 05:37:57.398533 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:57.398472    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6" event={"ID":"77b797a3-8d68-4859-a401-2f3e1a830fc3","Type":"ContainerDied","Data":"943e58b489ede0478e5ed6391315422b4f58885e4c8542c29a90c8180cf72473"}
Jul 17 05:37:57.398533 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:57.398504    2562 scope.go:117] "RemoveContainer" containerID="32a698461bee0a6447b520bb4ac5d668416539840edd774520151da5332b36ab"
Jul 17 05:37:57.408521 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:57.408500    2562 scope.go:117] "RemoveContainer" containerID="7bcb067e4d1a54718f3b94b4d47a400216c9b00a2b5e856d7ea2aea830d60995"
Jul 17 05:37:57.417299 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:57.417281    2562 scope.go:117] "RemoveContainer" containerID="32a698461bee0a6447b520bb4ac5d668416539840edd774520151da5332b36ab"
Jul 17 05:37:57.417591 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:37:57.417570    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32a698461bee0a6447b520bb4ac5d668416539840edd774520151da5332b36ab\": container with ID starting with 32a698461bee0a6447b520bb4ac5d668416539840edd774520151da5332b36ab not found: ID does not exist" containerID="32a698461bee0a6447b520bb4ac5d668416539840edd774520151da5332b36ab"
Jul 17 05:37:57.417652 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:57.417600    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32a698461bee0a6447b520bb4ac5d668416539840edd774520151da5332b36ab"} err="failed to get container status \"32a698461bee0a6447b520bb4ac5d668416539840edd774520151da5332b36ab\": rpc error: code = NotFound desc = could not find container \"32a698461bee0a6447b520bb4ac5d668416539840edd774520151da5332b36ab\": container with ID starting with 32a698461bee0a6447b520bb4ac5d668416539840edd774520151da5332b36ab not found: ID does not exist"
Jul 17 05:37:57.417652 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:57.417616    2562 scope.go:117] "RemoveContainer" containerID="7bcb067e4d1a54718f3b94b4d47a400216c9b00a2b5e856d7ea2aea830d60995"
Jul 17 05:37:57.417908 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:37:57.417888    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7bcb067e4d1a54718f3b94b4d47a400216c9b00a2b5e856d7ea2aea830d60995\": container with ID starting with 7bcb067e4d1a54718f3b94b4d47a400216c9b00a2b5e856d7ea2aea830d60995 not found: ID does not exist" containerID="7bcb067e4d1a54718f3b94b4d47a400216c9b00a2b5e856d7ea2aea830d60995"
Jul 17 05:37:57.417983 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:57.417914    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bcb067e4d1a54718f3b94b4d47a400216c9b00a2b5e856d7ea2aea830d60995"} err="failed to get container status \"7bcb067e4d1a54718f3b94b4d47a400216c9b00a2b5e856d7ea2aea830d60995\": rpc error: code = NotFound desc = could not find container \"7bcb067e4d1a54718f3b94b4d47a400216c9b00a2b5e856d7ea2aea830d60995\": container with ID starting with 7bcb067e4d1a54718f3b94b4d47a400216c9b00a2b5e856d7ea2aea830d60995 not found: ID does not exist"
Jul 17 05:37:57.422519 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:57.422489    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6"]
Jul 17 05:37:57.428430 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:57.428409    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-e1ab5d-openshift-default-85f4cb45-5kjs6"]
Jul 17 05:37:57.444839 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:37:57.444805    2562 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77b797a3_8d68_4859_a401_2f3e1a830fc3.slice/crio-943e58b489ede0478e5ed6391315422b4f58885e4c8542c29a90c8180cf72473\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77b797a3_8d68_4859_a401_2f3e1a830fc3.slice\": RecentStats: unable to find data in memory cache]"
Jul 17 05:37:58.889785 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:37:58.889747    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77b797a3-8d68-4859-a401-2f3e1a830fc3" path="/var/lib/kubelet/pods/77b797a3-8d68-4859-a401-2f3e1a830fc3/volumes"
Jul 17 05:38:02.871817 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:02.871686    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-63fb52-6f5cc68758-pbmph"]
Jul 17 05:38:02.872672 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:02.872634    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-63fb52-6f5cc68758-pbmph" podUID="4d013611-bf77-4fb2-bd72-e1458cf4c090" containerName="maas-api" containerID="cri-o://e0b0fef0b2a79662a082113116f8f83505b60b364fcbba2260ccd1a2370bada3" gracePeriod=30
Jul 17 05:38:03.136637 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:03.136607    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-63fb52-6f5cc68758-pbmph"
Jul 17 05:38:03.202101 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:03.202068    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l8kgz\" (UniqueName: \"kubernetes.io/projected/4d013611-bf77-4fb2-bd72-e1458cf4c090-kube-api-access-l8kgz\") pod \"4d013611-bf77-4fb2-bd72-e1458cf4c090\" (UID: \"4d013611-bf77-4fb2-bd72-e1458cf4c090\") "
Jul 17 05:38:03.202299 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:03.202113    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d013611-bf77-4fb2-bd72-e1458cf4c090-openshift-service-ca\") pod \"4d013611-bf77-4fb2-bd72-e1458cf4c090\" (UID: \"4d013611-bf77-4fb2-bd72-e1458cf4c090\") "
Jul 17 05:38:03.202299 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:03.202148    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d013611-bf77-4fb2-bd72-e1458cf4c090-maas-api-tls\") pod \"4d013611-bf77-4fb2-bd72-e1458cf4c090\" (UID: \"4d013611-bf77-4fb2-bd72-e1458cf4c090\") "
Jul 17 05:38:03.202500 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:03.202474    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d013611-bf77-4fb2-bd72-e1458cf4c090-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4d013611-bf77-4fb2-bd72-e1458cf4c090" (UID: "4d013611-bf77-4fb2-bd72-e1458cf4c090"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:38:03.204386 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:03.204359    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d013611-bf77-4fb2-bd72-e1458cf4c090-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4d013611-bf77-4fb2-bd72-e1458cf4c090" (UID: "4d013611-bf77-4fb2-bd72-e1458cf4c090"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 05:38:03.204492 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:03.204407    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d013611-bf77-4fb2-bd72-e1458cf4c090-kube-api-access-l8kgz" (OuterVolumeSpecName: "kube-api-access-l8kgz") pod "4d013611-bf77-4fb2-bd72-e1458cf4c090" (UID: "4d013611-bf77-4fb2-bd72-e1458cf4c090"). InnerVolumeSpecName "kube-api-access-l8kgz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:38:03.302991 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:03.302950    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8kgz\" (UniqueName: \"kubernetes.io/projected/4d013611-bf77-4fb2-bd72-e1458cf4c090-kube-api-access-l8kgz\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:38:03.302991 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:03.302989    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d013611-bf77-4fb2-bd72-e1458cf4c090-openshift-service-ca\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:38:03.303171 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:03.303000    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d013611-bf77-4fb2-bd72-e1458cf4c090-maas-api-tls\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:38:03.429298 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:03.429200    2562 generic.go:356] "Generic (PLEG): container finished" podID="4d013611-bf77-4fb2-bd72-e1458cf4c090" containerID="e0b0fef0b2a79662a082113116f8f83505b60b364fcbba2260ccd1a2370bada3" exitCode=0
Jul 17 05:38:03.429298 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:03.429244    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-63fb52-6f5cc68758-pbmph"
Jul 17 05:38:03.429298 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:03.429286    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-63fb52-6f5cc68758-pbmph" event={"ID":"4d013611-bf77-4fb2-bd72-e1458cf4c090","Type":"ContainerDied","Data":"e0b0fef0b2a79662a082113116f8f83505b60b364fcbba2260ccd1a2370bada3"}
Jul 17 05:38:03.429570 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:03.429345    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-63fb52-6f5cc68758-pbmph" event={"ID":"4d013611-bf77-4fb2-bd72-e1458cf4c090","Type":"ContainerDied","Data":"829bade22a3784cbfd8f8c7e86ddd7cd466a018bd54ca81d13cb02447154bf90"}
Jul 17 05:38:03.429570 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:03.429369    2562 scope.go:117] "RemoveContainer" containerID="e0b0fef0b2a79662a082113116f8f83505b60b364fcbba2260ccd1a2370bada3"
Jul 17 05:38:03.441683 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:03.441659    2562 scope.go:117] "RemoveContainer" containerID="e0b0fef0b2a79662a082113116f8f83505b60b364fcbba2260ccd1a2370bada3"
Jul 17 05:38:03.441924 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:38:03.441903    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0b0fef0b2a79662a082113116f8f83505b60b364fcbba2260ccd1a2370bada3\": container with ID starting with e0b0fef0b2a79662a082113116f8f83505b60b364fcbba2260ccd1a2370bada3 not found: ID does not exist" containerID="e0b0fef0b2a79662a082113116f8f83505b60b364fcbba2260ccd1a2370bada3"
Jul 17 05:38:03.442024 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:03.441931    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0b0fef0b2a79662a082113116f8f83505b60b364fcbba2260ccd1a2370bada3"} err="failed to get container status \"e0b0fef0b2a79662a082113116f8f83505b60b364fcbba2260ccd1a2370bada3\": rpc error: code = NotFound desc = could not find container \"e0b0fef0b2a79662a082113116f8f83505b60b364fcbba2260ccd1a2370bada3\": container with ID starting with e0b0fef0b2a79662a082113116f8f83505b60b364fcbba2260ccd1a2370bada3 not found: ID does not exist"
Jul 17 05:38:03.451798 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:03.451766    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-63fb52-6f5cc68758-pbmph"]
Jul 17 05:38:03.457229 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:03.457209    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-63fb52-6f5cc68758-pbmph"]
Jul 17 05:38:04.884304 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:04.884268    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d013611-bf77-4fb2-bd72-e1458cf4c090" path="/var/lib/kubelet/pods/4d013611-bf77-4fb2-bd72-e1458cf4c090/volumes"
Jul 17 05:38:18.092287 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:18.092242    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd"]
Jul 17 05:38:18.092733 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:18.092502    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd" podUID="1b56dd72-cddf-484d-8b14-b77e3a2d1c9e" containerName="istio-proxy" containerID="cri-o://88b031b2c3f0ae8308e1e1f3da8b29ce2691156a5d6a4f9f91f0445af6b6f0f6" gracePeriod=30
Jul 17 05:38:19.171163 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.171128    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"]
Jul 17 05:38:19.171554 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.171514    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0124abf8-492d-4fcf-85e5-e9659fe3be44" containerName="maas-api"
Jul 17 05:38:19.171554 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.171524    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0124abf8-492d-4fcf-85e5-e9659fe3be44" containerName="maas-api"
Jul 17 05:38:19.171554 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.171532    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5012f482-7044-464c-8309-7c473f1b5ded" containerName="revoke-keys"
Jul 17 05:38:19.171554 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.171538    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5012f482-7044-464c-8309-7c473f1b5ded" containerName="revoke-keys"
Jul 17 05:38:19.171554 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.171550    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77b797a3-8d68-4859-a401-2f3e1a830fc3" containerName="istio-proxy"
Jul 17 05:38:19.171554 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.171555    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="77b797a3-8d68-4859-a401-2f3e1a830fc3" containerName="istio-proxy"
Jul 17 05:38:19.171794 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.171563    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5012f482-7044-464c-8309-7c473f1b5ded" containerName="revoke-keys"
Jul 17 05:38:19.171794 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.171568    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5012f482-7044-464c-8309-7c473f1b5ded" containerName="revoke-keys"
Jul 17 05:38:19.171794 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.171575    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d013611-bf77-4fb2-bd72-e1458cf4c090" containerName="maas-api"
Jul 17 05:38:19.171794 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.171580    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d013611-bf77-4fb2-bd72-e1458cf4c090" containerName="maas-api"
Jul 17 05:38:19.171794 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.171586    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77b797a3-8d68-4859-a401-2f3e1a830fc3" containerName="istio-proxy"
Jul 17 05:38:19.171794 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.171591    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="77b797a3-8d68-4859-a401-2f3e1a830fc3" containerName="istio-proxy"
Jul 17 05:38:19.171794 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.171598    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5012f482-7044-464c-8309-7c473f1b5ded" containerName="revoke-keys"
Jul 17 05:38:19.171794 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.171603    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5012f482-7044-464c-8309-7c473f1b5ded" containerName="revoke-keys"
Jul 17 05:38:19.171794 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.171669    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5012f482-7044-464c-8309-7c473f1b5ded" containerName="revoke-keys"
Jul 17 05:38:19.171794 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.171678    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="77b797a3-8d68-4859-a401-2f3e1a830fc3" containerName="istio-proxy"
Jul 17 05:38:19.171794 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.171684    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d013611-bf77-4fb2-bd72-e1458cf4c090" containerName="maas-api"
Jul 17 05:38:19.171794 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.171692    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0124abf8-492d-4fcf-85e5-e9659fe3be44" containerName="maas-api"
Jul 17 05:38:19.171794 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.171700    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5012f482-7044-464c-8309-7c473f1b5ded" containerName="revoke-keys"
Jul 17 05:38:19.174968 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.174937    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"
Jul 17 05:38:19.177674 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.177653    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-938bd0-openshift-default-dockercfg-kvqjh\""
Jul 17 05:38:19.188051 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.188024    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"]
Jul 17 05:38:19.248466 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.248429    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4734a92-5798-4546-876e-96592d8964a4-istio-token\") pod \"e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8\" (UID: \"f4734a92-5798-4546-876e-96592d8964a4\") " pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"
Jul 17 05:38:19.248637 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.248497    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4734a92-5798-4546-876e-96592d8964a4-istiod-ca-cert\") pod \"e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8\" (UID: \"f4734a92-5798-4546-876e-96592d8964a4\") " pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"
Jul 17 05:38:19.248637 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.248612    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4734a92-5798-4546-876e-96592d8964a4-workload-certs\") pod \"e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8\" (UID: \"f4734a92-5798-4546-876e-96592d8964a4\") " pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"
Jul 17 05:38:19.248740 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.248644    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4734a92-5798-4546-876e-96592d8964a4-istio-envoy\") pod \"e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8\" (UID: \"f4734a92-5798-4546-876e-96592d8964a4\") " pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"
Jul 17 05:38:19.248740 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.248669    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4734a92-5798-4546-876e-96592d8964a4-workload-socket\") pod \"e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8\" (UID: \"f4734a92-5798-4546-876e-96592d8964a4\") " pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"
Jul 17 05:38:19.248740 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.248709    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4734a92-5798-4546-876e-96592d8964a4-istio-data\") pod \"e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8\" (UID: \"f4734a92-5798-4546-876e-96592d8964a4\") " pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"
Jul 17 05:38:19.248740 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.248724    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xvdz\" (UniqueName: \"kubernetes.io/projected/f4734a92-5798-4546-876e-96592d8964a4-kube-api-access-2xvdz\") pod \"e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8\" (UID: \"f4734a92-5798-4546-876e-96592d8964a4\") " pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"
Jul 17 05:38:19.248885 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.248753    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4734a92-5798-4546-876e-96592d8964a4-istio-podinfo\") pod \"e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8\" (UID: \"f4734a92-5798-4546-876e-96592d8964a4\") " pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"
Jul 17 05:38:19.248885 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.248776    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4734a92-5798-4546-876e-96592d8964a4-credential-socket\") pod \"e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8\" (UID: \"f4734a92-5798-4546-876e-96592d8964a4\") " pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"
Jul 17 05:38:19.349335 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.349293    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4734a92-5798-4546-876e-96592d8964a4-workload-certs\") pod \"e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8\" (UID: \"f4734a92-5798-4546-876e-96592d8964a4\") " pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"
Jul 17 05:38:19.349335 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.349334    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4734a92-5798-4546-876e-96592d8964a4-istio-envoy\") pod \"e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8\" (UID: \"f4734a92-5798-4546-876e-96592d8964a4\") " pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"
Jul 17 05:38:19.349605 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.349359    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4734a92-5798-4546-876e-96592d8964a4-workload-socket\") pod \"e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8\" (UID: \"f4734a92-5798-4546-876e-96592d8964a4\") " pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"
Jul 17 05:38:19.349605 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.349379    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4734a92-5798-4546-876e-96592d8964a4-istio-data\") pod \"e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8\" (UID: \"f4734a92-5798-4546-876e-96592d8964a4\") " pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"
Jul 17 05:38:19.349605 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.349397    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xvdz\" (UniqueName: \"kubernetes.io/projected/f4734a92-5798-4546-876e-96592d8964a4-kube-api-access-2xvdz\") pod \"e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8\" (UID: \"f4734a92-5798-4546-876e-96592d8964a4\") " pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"
Jul 17 05:38:19.349605 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.349424    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4734a92-5798-4546-876e-96592d8964a4-istio-podinfo\") pod \"e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8\" (UID: \"f4734a92-5798-4546-876e-96592d8964a4\") " pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"
Jul 17 05:38:19.349605 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.349448    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4734a92-5798-4546-876e-96592d8964a4-credential-socket\") pod \"e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8\" (UID: \"f4734a92-5798-4546-876e-96592d8964a4\") " pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"
Jul 17 05:38:19.349605 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.349484    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4734a92-5798-4546-876e-96592d8964a4-istio-token\") pod \"e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8\" (UID: \"f4734a92-5798-4546-876e-96592d8964a4\") " pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"
Jul 17 05:38:19.349605 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.349563    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4734a92-5798-4546-876e-96592d8964a4-istiod-ca-cert\") pod \"e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8\" (UID: \"f4734a92-5798-4546-876e-96592d8964a4\") " pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"
Jul 17 05:38:19.350059 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.349708    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4734a92-5798-4546-876e-96592d8964a4-workload-certs\") pod \"e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8\" (UID: \"f4734a92-5798-4546-876e-96592d8964a4\") " pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"
Jul 17 05:38:19.350059 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.349886    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4734a92-5798-4546-876e-96592d8964a4-istio-data\") pod \"e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8\" (UID: \"f4734a92-5798-4546-876e-96592d8964a4\") " pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"
Jul 17 05:38:19.350059 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.350043    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4734a92-5798-4546-876e-96592d8964a4-credential-socket\") pod \"e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8\" (UID: \"f4734a92-5798-4546-876e-96592d8964a4\") " pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"
Jul 17 05:38:19.350228 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.350194    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4734a92-5798-4546-876e-96592d8964a4-workload-socket\") pod \"e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8\" (UID: \"f4734a92-5798-4546-876e-96592d8964a4\") " pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"
Jul 17 05:38:19.350451 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.350322    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4734a92-5798-4546-876e-96592d8964a4-istiod-ca-cert\") pod \"e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8\" (UID: \"f4734a92-5798-4546-876e-96592d8964a4\") " pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"
Jul 17 05:38:19.351929 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.351906    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4734a92-5798-4546-876e-96592d8964a4-istio-envoy\") pod \"e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8\" (UID: \"f4734a92-5798-4546-876e-96592d8964a4\") " pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"
Jul 17 05:38:19.352287 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.352265    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4734a92-5798-4546-876e-96592d8964a4-istio-podinfo\") pod \"e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8\" (UID: \"f4734a92-5798-4546-876e-96592d8964a4\") " pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"
Jul 17 05:38:19.358911 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.358887    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4734a92-5798-4546-876e-96592d8964a4-istio-token\") pod \"e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8\" (UID: \"f4734a92-5798-4546-876e-96592d8964a4\") " pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"
Jul 17 05:38:19.359100 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.359050    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xvdz\" (UniqueName: \"kubernetes.io/projected/f4734a92-5798-4546-876e-96592d8964a4-kube-api-access-2xvdz\") pod \"e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8\" (UID: \"f4734a92-5798-4546-876e-96592d8964a4\") " pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"
Jul 17 05:38:19.486027 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.485936    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"
Jul 17 05:38:19.635754 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.635714    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"]
Jul 17 05:38:19.637218 ip-10-0-132-19 kubenswrapper[2562]: W0717 05:38:19.637189    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4734a92_5798_4546_876e_96592d8964a4.slice/crio-f81fc2adccf8ed58a4c8f28c9e14f9d96dfc754ec12f4b52e1d4dfda2fe220a5 WatchSource:0}: Error finding container f81fc2adccf8ed58a4c8f28c9e14f9d96dfc754ec12f4b52e1d4dfda2fe220a5: Status 404 returned error can't find the container with id f81fc2adccf8ed58a4c8f28c9e14f9d96dfc754ec12f4b52e1d4dfda2fe220a5
Jul 17 05:38:19.639506 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.639471    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:38:19.639595 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.639538    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:38:19.639595 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:19.639567    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:38:20.523450 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:20.523413    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8" event={"ID":"f4734a92-5798-4546-876e-96592d8964a4","Type":"ContainerStarted","Data":"cb1c9062729f7e87e08c8b7209c23521e628222f0c29a4dbd630f6344223e97d"}
Jul 17 05:38:20.523873 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:20.523474    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8" event={"ID":"f4734a92-5798-4546-876e-96592d8964a4","Type":"ContainerStarted","Data":"f81fc2adccf8ed58a4c8f28c9e14f9d96dfc754ec12f4b52e1d4dfda2fe220a5"}
Jul 17 05:38:20.545160 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:20.545106    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8" podStartSLOduration=1.545089731 podStartE2EDuration="1.545089731s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:38:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:38:20.543986288 +0000 UTC m=+2888.227579959" watchObservedRunningTime="2026-07-17 05:38:20.545089731 +0000 UTC m=+2888.228683451"
Jul 17 05:38:21.487012 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:21.486976    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"
Jul 17 05:38:21.492049 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:21.492029    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"
Jul 17 05:38:21.528920 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:21.528885    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"
Jul 17 05:38:21.529762 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:21.529744    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"
Jul 17 05:38:22.621550 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:22.621515    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-938bd0-78d6b65c78-8mqqx"]
Jul 17 05:38:22.622289 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:22.622270    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5012f482-7044-464c-8309-7c473f1b5ded" containerName="revoke-keys"
Jul 17 05:38:22.622340 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:22.622296    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="77b797a3-8d68-4859-a401-2f3e1a830fc3" containerName="istio-proxy"
Jul 17 05:38:22.625769 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:22.625750    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-938bd0-78d6b65c78-8mqqx"
Jul 17 05:38:22.629120 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:22.629101    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-xwcd7\""
Jul 17 05:38:22.629242 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:22.629150    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-938bd0-serving-cert\""
Jul 17 05:38:22.638377 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:22.638348    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-938bd0-78d6b65c78-8mqqx"]
Jul 17 05:38:22.685570 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:22.685533    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flt7p\" (UniqueName: \"kubernetes.io/projected/b7ef57ed-1edc-4347-8369-5c9ab1748008-kube-api-access-flt7p\") pod \"maas-api-e2e-api-a-938bd0-78d6b65c78-8mqqx\" (UID: \"b7ef57ed-1edc-4347-8369-5c9ab1748008\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-938bd0-78d6b65c78-8mqqx"
Jul 17 05:38:22.685731 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:22.685610    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7ef57ed-1edc-4347-8369-5c9ab1748008-maas-api-tls\") pod \"maas-api-e2e-api-a-938bd0-78d6b65c78-8mqqx\" (UID: \"b7ef57ed-1edc-4347-8369-5c9ab1748008\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-938bd0-78d6b65c78-8mqqx"
Jul 17 05:38:22.685731 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:22.685662    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7ef57ed-1edc-4347-8369-5c9ab1748008-openshift-service-ca\") pod \"maas-api-e2e-api-a-938bd0-78d6b65c78-8mqqx\" (UID: \"b7ef57ed-1edc-4347-8369-5c9ab1748008\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-938bd0-78d6b65c78-8mqqx"
Jul 17 05:38:22.786679 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:22.786640    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7ef57ed-1edc-4347-8369-5c9ab1748008-openshift-service-ca\") pod \"maas-api-e2e-api-a-938bd0-78d6b65c78-8mqqx\" (UID: \"b7ef57ed-1edc-4347-8369-5c9ab1748008\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-938bd0-78d6b65c78-8mqqx"
Jul 17 05:38:22.786889 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:22.786712    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flt7p\" (UniqueName: \"kubernetes.io/projected/b7ef57ed-1edc-4347-8369-5c9ab1748008-kube-api-access-flt7p\") pod \"maas-api-e2e-api-a-938bd0-78d6b65c78-8mqqx\" (UID: \"b7ef57ed-1edc-4347-8369-5c9ab1748008\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-938bd0-78d6b65c78-8mqqx"
Jul 17 05:38:22.786889 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:22.786766    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7ef57ed-1edc-4347-8369-5c9ab1748008-maas-api-tls\") pod \"maas-api-e2e-api-a-938bd0-78d6b65c78-8mqqx\" (UID: \"b7ef57ed-1edc-4347-8369-5c9ab1748008\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-938bd0-78d6b65c78-8mqqx"
Jul 17 05:38:22.787377 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:22.787350    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7ef57ed-1edc-4347-8369-5c9ab1748008-openshift-service-ca\") pod \"maas-api-e2e-api-a-938bd0-78d6b65c78-8mqqx\" (UID: \"b7ef57ed-1edc-4347-8369-5c9ab1748008\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-938bd0-78d6b65c78-8mqqx"
Jul 17 05:38:22.789436 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:22.789414    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7ef57ed-1edc-4347-8369-5c9ab1748008-maas-api-tls\") pod \"maas-api-e2e-api-a-938bd0-78d6b65c78-8mqqx\" (UID: \"b7ef57ed-1edc-4347-8369-5c9ab1748008\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-938bd0-78d6b65c78-8mqqx"
Jul 17 05:38:22.797773 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:22.797751    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-flt7p\" (UniqueName: \"kubernetes.io/projected/b7ef57ed-1edc-4347-8369-5c9ab1748008-kube-api-access-flt7p\") pod \"maas-api-e2e-api-a-938bd0-78d6b65c78-8mqqx\" (UID: \"b7ef57ed-1edc-4347-8369-5c9ab1748008\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-938bd0-78d6b65c78-8mqqx"
Jul 17 05:38:22.938221 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:22.938122    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-938bd0-78d6b65c78-8mqqx"
Jul 17 05:38:23.078879 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:23.078853    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-938bd0-78d6b65c78-8mqqx"]
Jul 17 05:38:23.080982 ip-10-0-132-19 kubenswrapper[2562]: W0717 05:38:23.080937    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7ef57ed_1edc_4347_8369_5c9ab1748008.slice/crio-69c67ae27b6349f13a0184536aefba24f14fc672fcb59e2fdafd9c784d20423f WatchSource:0}: Error finding container 69c67ae27b6349f13a0184536aefba24f14fc672fcb59e2fdafd9c784d20423f: Status 404 returned error can't find the container with id 69c67ae27b6349f13a0184536aefba24f14fc672fcb59e2fdafd9c784d20423f
Jul 17 05:38:23.082778 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:23.082757    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 05:38:23.331811 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:23.331785    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd"
Jul 17 05:38:23.390717 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:23.390682    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-84pb5\" (UniqueName: \"kubernetes.io/projected/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-kube-api-access-84pb5\") pod \"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e\" (UID: \"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e\") "
Jul 17 05:38:23.390918 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:23.390758    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-istio-data\") pod \"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e\" (UID: \"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e\") "
Jul 17 05:38:23.390918 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:23.390804    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-workload-certs\") pod \"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e\" (UID: \"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e\") "
Jul 17 05:38:23.390918 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:23.390842    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-istio-token\") pod \"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e\" (UID: \"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e\") "
Jul 17 05:38:23.391120 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:23.390978    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-credential-socket\") pod \"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e\" (UID: \"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e\") "
Jul 17 05:38:23.391120 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:23.391013    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-istio-podinfo\") pod \"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e\" (UID: \"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e\") "
Jul 17 05:38:23.391120 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:23.391046    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-istiod-ca-cert\") pod \"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e\" (UID: \"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e\") "
Jul 17 05:38:23.391120 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:23.391081    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-workload-socket\") pod \"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e\" (UID: \"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e\") "
Jul 17 05:38:23.391333 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:23.391121    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-istio-envoy\") pod \"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e\" (UID: \"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e\") "
Jul 17 05:38:23.391856 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:23.391387    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1b56dd72-cddf-484d-8b14-b77e3a2d1c9e" (UID: "1b56dd72-cddf-484d-8b14-b77e3a2d1c9e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:38:23.391856 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:23.391612    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1b56dd72-cddf-484d-8b14-b77e3a2d1c9e" (UID: "1b56dd72-cddf-484d-8b14-b77e3a2d1c9e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:38:23.391856 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:23.391689    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-credential-socket\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:38:23.391856 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:23.391714    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1b56dd72-cddf-484d-8b14-b77e3a2d1c9e" (UID: "1b56dd72-cddf-484d-8b14-b77e3a2d1c9e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:38:23.391856 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:23.391739    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-istio-data" (OuterVolumeSpecName: "istio-data") pod "1b56dd72-cddf-484d-8b14-b77e3a2d1c9e" (UID: "1b56dd72-cddf-484d-8b14-b77e3a2d1c9e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:38:23.391856 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:23.391813    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1b56dd72-cddf-484d-8b14-b77e3a2d1c9e" (UID: "1b56dd72-cddf-484d-8b14-b77e3a2d1c9e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:38:23.394166 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:23.394134    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-kube-api-access-84pb5" (OuterVolumeSpecName: "kube-api-access-84pb5") pod "1b56dd72-cddf-484d-8b14-b77e3a2d1c9e" (UID: "1b56dd72-cddf-484d-8b14-b77e3a2d1c9e"). InnerVolumeSpecName "kube-api-access-84pb5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:38:23.394282 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:23.394258    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1b56dd72-cddf-484d-8b14-b77e3a2d1c9e" (UID: "1b56dd72-cddf-484d-8b14-b77e3a2d1c9e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 05:38:23.394344 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:23.394313    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1b56dd72-cddf-484d-8b14-b77e3a2d1c9e" (UID: "1b56dd72-cddf-484d-8b14-b77e3a2d1c9e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:38:23.394398 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:23.394382    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-istio-token" (OuterVolumeSpecName: "istio-token") pod "1b56dd72-cddf-484d-8b14-b77e3a2d1c9e" (UID: "1b56dd72-cddf-484d-8b14-b77e3a2d1c9e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:38:23.493151 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:23.493076    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-istio-podinfo\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:38:23.493151 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:23.493102    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-istiod-ca-cert\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:38:23.493151 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:23.493110    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-workload-socket\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:38:23.493151 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:23.493118    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-istio-envoy\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:38:23.493151 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:23.493126    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84pb5\" (UniqueName: \"kubernetes.io/projected/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-kube-api-access-84pb5\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:38:23.493151 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:23.493136    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-istio-data\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:38:23.493151 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:23.493145    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-workload-certs\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:38:23.493151 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:23.493152    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e-istio-token\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:38:23.544635 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:23.544603    2562 generic.go:356] "Generic (PLEG): container finished" podID="1b56dd72-cddf-484d-8b14-b77e3a2d1c9e" containerID="88b031b2c3f0ae8308e1e1f3da8b29ce2691156a5d6a4f9f91f0445af6b6f0f6" exitCode=0
Jul 17 05:38:23.544744 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:23.544661    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd"
Jul 17 05:38:23.544744 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:23.544725    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd" event={"ID":"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e","Type":"ContainerDied","Data":"88b031b2c3f0ae8308e1e1f3da8b29ce2691156a5d6a4f9f91f0445af6b6f0f6"}
Jul 17 05:38:23.544864 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:23.544782    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd" event={"ID":"1b56dd72-cddf-484d-8b14-b77e3a2d1c9e","Type":"ContainerDied","Data":"306861ee318491d32d1b78e6a5e84b0245c2a81ea90544fe258bb141d3f975f7"}
Jul 17 05:38:23.544864 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:23.544809    2562 scope.go:117] "RemoveContainer" containerID="88b031b2c3f0ae8308e1e1f3da8b29ce2691156a5d6a4f9f91f0445af6b6f0f6"
Jul 17 05:38:23.546369 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:23.546341    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-938bd0-78d6b65c78-8mqqx" event={"ID":"b7ef57ed-1edc-4347-8369-5c9ab1748008","Type":"ContainerStarted","Data":"69c67ae27b6349f13a0184536aefba24f14fc672fcb59e2fdafd9c784d20423f"}
Jul 17 05:38:23.563334 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:23.563315    2562 scope.go:117] "RemoveContainer" containerID="f16b8abd219c07a375503f9178c56f487da48aa16a21f1dcd557a334fcc20100"
Jul 17 05:38:23.578396 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:23.578371    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd"]
Jul 17 05:38:23.583223 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:23.583199    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-63fb52-openshift-default-577bc597cf-ws8qd"]
Jul 17 05:38:23.583513 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:23.583486    2562 scope.go:117] "RemoveContainer" containerID="88b031b2c3f0ae8308e1e1f3da8b29ce2691156a5d6a4f9f91f0445af6b6f0f6"
Jul 17 05:38:23.583842 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:38:23.583822    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88b031b2c3f0ae8308e1e1f3da8b29ce2691156a5d6a4f9f91f0445af6b6f0f6\": container with ID starting with 88b031b2c3f0ae8308e1e1f3da8b29ce2691156a5d6a4f9f91f0445af6b6f0f6 not found: ID does not exist" containerID="88b031b2c3f0ae8308e1e1f3da8b29ce2691156a5d6a4f9f91f0445af6b6f0f6"
Jul 17 05:38:23.583897 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:23.583851    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88b031b2c3f0ae8308e1e1f3da8b29ce2691156a5d6a4f9f91f0445af6b6f0f6"} err="failed to get container status \"88b031b2c3f0ae8308e1e1f3da8b29ce2691156a5d6a4f9f91f0445af6b6f0f6\": rpc error: code = NotFound desc = could not find container \"88b031b2c3f0ae8308e1e1f3da8b29ce2691156a5d6a4f9f91f0445af6b6f0f6\": container with ID starting with 88b031b2c3f0ae8308e1e1f3da8b29ce2691156a5d6a4f9f91f0445af6b6f0f6 not found: ID does not exist"
Jul 17 05:38:23.583897 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:23.583869    2562 scope.go:117] "RemoveContainer" containerID="f16b8abd219c07a375503f9178c56f487da48aa16a21f1dcd557a334fcc20100"
Jul 17 05:38:23.584222 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:38:23.584205    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f16b8abd219c07a375503f9178c56f487da48aa16a21f1dcd557a334fcc20100\": container with ID starting with f16b8abd219c07a375503f9178c56f487da48aa16a21f1dcd557a334fcc20100 not found: ID does not exist" containerID="f16b8abd219c07a375503f9178c56f487da48aa16a21f1dcd557a334fcc20100"
Jul 17 05:38:23.584278 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:23.584229    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f16b8abd219c07a375503f9178c56f487da48aa16a21f1dcd557a334fcc20100"} err="failed to get container status \"f16b8abd219c07a375503f9178c56f487da48aa16a21f1dcd557a334fcc20100\": rpc error: code = NotFound desc = could not find container \"f16b8abd219c07a375503f9178c56f487da48aa16a21f1dcd557a334fcc20100\": container with ID starting with f16b8abd219c07a375503f9178c56f487da48aa16a21f1dcd557a334fcc20100 not found: ID does not exist"
Jul 17 05:38:24.553764 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:24.553726    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-938bd0-78d6b65c78-8mqqx" event={"ID":"b7ef57ed-1edc-4347-8369-5c9ab1748008","Type":"ContainerStarted","Data":"1042a8d3a67a9d06c2bcd196e1b91c64da37566f2e563dbb28e0dfb13a27cc4d"}
Jul 17 05:38:24.554161 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:24.553861    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-938bd0-78d6b65c78-8mqqx"
Jul 17 05:38:24.610630 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:24.610583    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-938bd0-78d6b65c78-8mqqx" podStartSLOduration=2.167026605 podStartE2EDuration="2.610567319s" totalImagesPullingTime="443.540714ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:38:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:38:24.608849489 +0000 UTC m=+2892.292443169" watchObservedRunningTime="2026-07-17 05:38:24.610567319 +0000 UTC m=+2892.294160975"
Jul 17 05:38:24.885175 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:24.885140    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b56dd72-cddf-484d-8b14-b77e3a2d1c9e" path="/var/lib/kubelet/pods/1b56dd72-cddf-484d-8b14-b77e3a2d1c9e/volumes"
Jul 17 05:38:27.822164 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:27.822130    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s"]
Jul 17 05:38:27.822727 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:27.822656    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b56dd72-cddf-484d-8b14-b77e3a2d1c9e" containerName="istio-proxy"
Jul 17 05:38:27.822727 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:27.822675    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b56dd72-cddf-484d-8b14-b77e3a2d1c9e" containerName="istio-proxy"
Jul 17 05:38:27.822727 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:27.822686    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b56dd72-cddf-484d-8b14-b77e3a2d1c9e" containerName="istio-proxy"
Jul 17 05:38:27.822727 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:27.822695    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b56dd72-cddf-484d-8b14-b77e3a2d1c9e" containerName="istio-proxy"
Jul 17 05:38:27.823006 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:27.822824    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b56dd72-cddf-484d-8b14-b77e3a2d1c9e" containerName="istio-proxy"
Jul 17 05:38:27.823006 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:27.822842    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b56dd72-cddf-484d-8b14-b77e3a2d1c9e" containerName="istio-proxy"
Jul 17 05:38:27.825885 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:27.825866    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s"
Jul 17 05:38:27.828728 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:27.828708    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-ce2966-openshift-default-dockercfg-c556c\""
Jul 17 05:38:27.845085 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:27.845059    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s"]
Jul 17 05:38:27.937509 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:27.937469    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7190c888-53d0-428e-901a-f8d87456541c-istio-podinfo\") pod \"e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s\" (UID: \"7190c888-53d0-428e-901a-f8d87456541c\") " pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s"
Jul 17 05:38:27.937769 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:27.937743    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skrs9\" (UniqueName: \"kubernetes.io/projected/7190c888-53d0-428e-901a-f8d87456541c-kube-api-access-skrs9\") pod \"e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s\" (UID: \"7190c888-53d0-428e-901a-f8d87456541c\") " pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s"
Jul 17 05:38:27.937941 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:27.937918    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7190c888-53d0-428e-901a-f8d87456541c-istio-token\") pod \"e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s\" (UID: \"7190c888-53d0-428e-901a-f8d87456541c\") " pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s"
Jul 17 05:38:27.938101 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:27.938084    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7190c888-53d0-428e-901a-f8d87456541c-istio-data\") pod \"e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s\" (UID: \"7190c888-53d0-428e-901a-f8d87456541c\") " pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s"
Jul 17 05:38:27.938307 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:27.938249    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7190c888-53d0-428e-901a-f8d87456541c-credential-socket\") pod \"e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s\" (UID: \"7190c888-53d0-428e-901a-f8d87456541c\") " pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s"
Jul 17 05:38:27.938430 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:27.938415    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7190c888-53d0-428e-901a-f8d87456541c-istiod-ca-cert\") pod \"e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s\" (UID: \"7190c888-53d0-428e-901a-f8d87456541c\") " pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s"
Jul 17 05:38:27.938543 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:27.938530    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7190c888-53d0-428e-901a-f8d87456541c-workload-socket\") pod \"e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s\" (UID: \"7190c888-53d0-428e-901a-f8d87456541c\") " pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s"
Jul 17 05:38:27.938637 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:27.938625    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7190c888-53d0-428e-901a-f8d87456541c-workload-certs\") pod \"e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s\" (UID: \"7190c888-53d0-428e-901a-f8d87456541c\") " pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s"
Jul 17 05:38:27.938781 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:27.938768    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7190c888-53d0-428e-901a-f8d87456541c-istio-envoy\") pod \"e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s\" (UID: \"7190c888-53d0-428e-901a-f8d87456541c\") " pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s"
Jul 17 05:38:28.040353 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:28.040322    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7190c888-53d0-428e-901a-f8d87456541c-istio-podinfo\") pod \"e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s\" (UID: \"7190c888-53d0-428e-901a-f8d87456541c\") " pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s"
Jul 17 05:38:28.040549 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:28.040361    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skrs9\" (UniqueName: \"kubernetes.io/projected/7190c888-53d0-428e-901a-f8d87456541c-kube-api-access-skrs9\") pod \"e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s\" (UID: \"7190c888-53d0-428e-901a-f8d87456541c\") " pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s"
Jul 17 05:38:28.040549 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:28.040419    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7190c888-53d0-428e-901a-f8d87456541c-istio-token\") pod \"e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s\" (UID: \"7190c888-53d0-428e-901a-f8d87456541c\") " pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s"
Jul 17 05:38:28.040549 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:28.040448    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7190c888-53d0-428e-901a-f8d87456541c-istio-data\") pod \"e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s\" (UID: \"7190c888-53d0-428e-901a-f8d87456541c\") " pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s"
Jul 17 05:38:28.040549 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:28.040493    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7190c888-53d0-428e-901a-f8d87456541c-credential-socket\") pod \"e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s\" (UID: \"7190c888-53d0-428e-901a-f8d87456541c\") " pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s"
Jul 17 05:38:28.040549 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:28.040534    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7190c888-53d0-428e-901a-f8d87456541c-istiod-ca-cert\") pod \"e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s\" (UID: \"7190c888-53d0-428e-901a-f8d87456541c\") " pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s"
Jul 17 05:38:28.040825 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:28.040566    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7190c888-53d0-428e-901a-f8d87456541c-workload-socket\") pod \"e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s\" (UID: \"7190c888-53d0-428e-901a-f8d87456541c\") " pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s"
Jul 17 05:38:28.040825 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:28.040590    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7190c888-53d0-428e-901a-f8d87456541c-workload-certs\") pod \"e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s\" (UID: \"7190c888-53d0-428e-901a-f8d87456541c\") " pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s"
Jul 17 05:38:28.040825 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:28.040614    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7190c888-53d0-428e-901a-f8d87456541c-istio-envoy\") pod \"e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s\" (UID: \"7190c888-53d0-428e-901a-f8d87456541c\") " pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s"
Jul 17 05:38:28.041005 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:28.040935    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7190c888-53d0-428e-901a-f8d87456541c-istio-data\") pod \"e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s\" (UID: \"7190c888-53d0-428e-901a-f8d87456541c\") " pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s"
Jul 17 05:38:28.041076 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:28.041002    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7190c888-53d0-428e-901a-f8d87456541c-workload-socket\") pod \"e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s\" (UID: \"7190c888-53d0-428e-901a-f8d87456541c\") " pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s"
Jul 17 05:38:28.041076 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:28.041023    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7190c888-53d0-428e-901a-f8d87456541c-workload-certs\") pod \"e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s\" (UID: \"7190c888-53d0-428e-901a-f8d87456541c\") " pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s"
Jul 17 05:38:28.041408 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:28.041236    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7190c888-53d0-428e-901a-f8d87456541c-credential-socket\") pod \"e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s\" (UID: \"7190c888-53d0-428e-901a-f8d87456541c\") " pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s"
Jul 17 05:38:28.041408 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:28.041387    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7190c888-53d0-428e-901a-f8d87456541c-istiod-ca-cert\") pod \"e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s\" (UID: \"7190c888-53d0-428e-901a-f8d87456541c\") " pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s"
Jul 17 05:38:28.042994 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:28.042957    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7190c888-53d0-428e-901a-f8d87456541c-istio-envoy\") pod \"e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s\" (UID: \"7190c888-53d0-428e-901a-f8d87456541c\") " pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s"
Jul 17 05:38:28.043118 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:28.043102    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7190c888-53d0-428e-901a-f8d87456541c-istio-podinfo\") pod \"e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s\" (UID: \"7190c888-53d0-428e-901a-f8d87456541c\") " pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s"
Jul 17 05:38:28.048641 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:28.048619    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7190c888-53d0-428e-901a-f8d87456541c-istio-token\") pod \"e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s\" (UID: \"7190c888-53d0-428e-901a-f8d87456541c\") " pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s"
Jul 17 05:38:28.048727 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:28.048675    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-skrs9\" (UniqueName: \"kubernetes.io/projected/7190c888-53d0-428e-901a-f8d87456541c-kube-api-access-skrs9\") pod \"e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s\" (UID: \"7190c888-53d0-428e-901a-f8d87456541c\") " pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s"
Jul 17 05:38:28.137628 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:28.137524    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s"
Jul 17 05:38:28.333512 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:28.333461    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s"]
Jul 17 05:38:28.335866 ip-10-0-132-19 kubenswrapper[2562]: W0717 05:38:28.335826    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7190c888_53d0_428e_901a_f8d87456541c.slice/crio-cd6e18d05861891cbad574e586bb47609a10a5f76846eb8e2078d3ce5d70361b WatchSource:0}: Error finding container cd6e18d05861891cbad574e586bb47609a10a5f76846eb8e2078d3ce5d70361b: Status 404 returned error can't find the container with id cd6e18d05861891cbad574e586bb47609a10a5f76846eb8e2078d3ce5d70361b
Jul 17 05:38:28.338315 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:28.338284    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:38:28.338398 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:28.338350    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:38:28.338398 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:28.338380    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:38:28.594620 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:28.594577    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s" event={"ID":"7190c888-53d0-428e-901a-f8d87456541c","Type":"ContainerStarted","Data":"6ee7411a8ed8c84ea2633f944e1c18620b2ac2ece470c23ea13719b22ea4643c"}
Jul 17 05:38:28.594818 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:28.594663    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s" event={"ID":"7190c888-53d0-428e-901a-f8d87456541c","Type":"ContainerStarted","Data":"cd6e18d05861891cbad574e586bb47609a10a5f76846eb8e2078d3ce5d70361b"}
Jul 17 05:38:28.631876 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:28.631815    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s" podStartSLOduration=1.6317951160000002 podStartE2EDuration="1.631795116s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:38:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:38:28.629320712 +0000 UTC m=+2896.312914396" watchObservedRunningTime="2026-07-17 05:38:28.631795116 +0000 UTC m=+2896.315388770"
Jul 17 05:38:29.137732 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:29.137694    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s"
Jul 17 05:38:29.142804 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:29.142779    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s"
Jul 17 05:38:29.599509 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:29.599467    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s"
Jul 17 05:38:29.600571 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:29.600549    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s"
Jul 17 05:38:30.571804 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:30.571762    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-938bd0-78d6b65c78-8mqqx"
Jul 17 05:38:35.635425 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:35.635388    2562 generic.go:356] "Generic (PLEG): container finished" podID="f4734a92-5798-4546-876e-96592d8964a4" containerID="cb1c9062729f7e87e08c8b7209c23521e628222f0c29a4dbd630f6344223e97d" exitCode=0
Jul 17 05:38:35.635839 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:35.635462    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8" event={"ID":"f4734a92-5798-4546-876e-96592d8964a4","Type":"ContainerDied","Data":"cb1c9062729f7e87e08c8b7209c23521e628222f0c29a4dbd630f6344223e97d"}
Jul 17 05:38:35.635914 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:35.635899    2562 scope.go:117] "RemoveContainer" containerID="cb1c9062729f7e87e08c8b7209c23521e628222f0c29a4dbd630f6344223e97d"
Jul 17 05:38:35.636914 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:35.636883    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:38:35.637043 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:35.636986    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:38:35.637109 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:35.637040    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:38:36.529296 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:36.529249    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"
Jul 17 05:38:36.641806 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:36.641772    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8" event={"ID":"f4734a92-5798-4546-876e-96592d8964a4","Type":"ContainerStarted","Data":"eeb1ef4e59ecc568be79b8198177deb49f3ba303a1838b7f41b7068eb7dd9341"}
Jul 17 05:38:37.487163 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:37.487122    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"
Jul 17 05:38:37.488449 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:37.488422    2562 patch_prober.go:28] interesting pod/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Jul 17 05:38:37.488586 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:37.488471    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8" podUID="f4734a92-5798-4546-876e-96592d8964a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Jul 17 05:38:38.486552 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:38.486516    2562 patch_prober.go:28] interesting pod/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Jul 17 05:38:38.486995 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:38.486569    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8" podUID="f4734a92-5798-4546-876e-96592d8964a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Jul 17 05:38:39.487443 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:39.487394    2562 patch_prober.go:28] interesting pod/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Jul 17 05:38:39.488001 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:39.487466    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8" podUID="f4734a92-5798-4546-876e-96592d8964a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Jul 17 05:38:40.486798 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:40.486756    2562 patch_prober.go:28] interesting pod/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Jul 17 05:38:40.487142 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:40.487109    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8" podUID="f4734a92-5798-4546-876e-96592d8964a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Jul 17 05:38:41.490782 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:41.490751    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"
Jul 17 05:38:41.491212 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:41.491002    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"
Jul 17 05:38:41.491789 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:41.491772    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"
Jul 17 05:38:45.641109 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:45.641076    2562 patch_prober.go:28] interesting pod/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 17 05:38:45.641517 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:45.641126    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s" podUID="7190c888-53d0-428e-901a-f8d87456541c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 17 05:38:46.706588 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:46.706549    2562 generic.go:356] "Generic (PLEG): container finished" podID="7190c888-53d0-428e-901a-f8d87456541c" containerID="6ee7411a8ed8c84ea2633f944e1c18620b2ac2ece470c23ea13719b22ea4643c" exitCode=0
Jul 17 05:38:46.707049 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:46.706626    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s" event={"ID":"7190c888-53d0-428e-901a-f8d87456541c","Type":"ContainerDied","Data":"6ee7411a8ed8c84ea2633f944e1c18620b2ac2ece470c23ea13719b22ea4643c"}
Jul 17 05:38:46.707100 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:46.707067    2562 scope.go:117] "RemoveContainer" containerID="6ee7411a8ed8c84ea2633f944e1c18620b2ac2ece470c23ea13719b22ea4643c"
Jul 17 05:38:46.708046 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:46.708020    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:38:46.708103 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:46.708083    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:38:46.708140 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:46.708116    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:38:47.713092 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:47.713055    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s" event={"ID":"7190c888-53d0-428e-901a-f8d87456541c","Type":"ContainerStarted","Data":"55657eeb5605df305bfb6a80ee75bc05b784696c905088134fbdad244e290401"}
Jul 17 05:38:48.137900 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:48.137868    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s"
Jul 17 05:38:48.138135 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:48.138074    2562 patch_prober.go:28] interesting pod/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Jul 17 05:38:48.138190 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:48.138132    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s" podUID="7190c888-53d0-428e-901a-f8d87456541c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Jul 17 05:38:49.138046 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:49.138012    2562 patch_prober.go:28] interesting pod/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Jul 17 05:38:49.138433 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:49.138062    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s" podUID="7190c888-53d0-428e-901a-f8d87456541c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Jul 17 05:38:50.138900 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:50.138866    2562 patch_prober.go:28] interesting pod/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Jul 17 05:38:50.139323 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:50.138934    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s" podUID="7190c888-53d0-428e-901a-f8d87456541c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Jul 17 05:38:51.138131 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:51.138090    2562 patch_prober.go:28] interesting pod/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Jul 17 05:38:51.138326 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:51.138153    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s" podUID="7190c888-53d0-428e-901a-f8d87456541c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Jul 17 05:38:52.142178 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:52.142145    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s"
Jul 17 05:38:52.142665 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:52.142523    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s"
Jul 17 05:38:52.143111 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:52.143094    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s"
Jul 17 05:38:54.087730 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:54.087693    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-938bd0-78d6b65c78-8mqqx"]
Jul 17 05:38:54.088257 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:54.087932    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-938bd0-78d6b65c78-8mqqx" podUID="b7ef57ed-1edc-4347-8369-5c9ab1748008" containerName="maas-api" containerID="cri-o://1042a8d3a67a9d06c2bcd196e1b91c64da37566f2e563dbb28e0dfb13a27cc4d" gracePeriod=30
Jul 17 05:38:54.340717 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:54.340661    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-938bd0-78d6b65c78-8mqqx"
Jul 17 05:38:54.447699 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:54.447673    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-flt7p\" (UniqueName: \"kubernetes.io/projected/b7ef57ed-1edc-4347-8369-5c9ab1748008-kube-api-access-flt7p\") pod \"b7ef57ed-1edc-4347-8369-5c9ab1748008\" (UID: \"b7ef57ed-1edc-4347-8369-5c9ab1748008\") "
Jul 17 05:38:54.447860 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:54.447725    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7ef57ed-1edc-4347-8369-5c9ab1748008-openshift-service-ca\") pod \"b7ef57ed-1edc-4347-8369-5c9ab1748008\" (UID: \"b7ef57ed-1edc-4347-8369-5c9ab1748008\") "
Jul 17 05:38:54.447860 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:54.447795    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7ef57ed-1edc-4347-8369-5c9ab1748008-maas-api-tls\") pod \"b7ef57ed-1edc-4347-8369-5c9ab1748008\" (UID: \"b7ef57ed-1edc-4347-8369-5c9ab1748008\") "
Jul 17 05:38:54.448165 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:54.448135    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7ef57ed-1edc-4347-8369-5c9ab1748008-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b7ef57ed-1edc-4347-8369-5c9ab1748008" (UID: "b7ef57ed-1edc-4347-8369-5c9ab1748008"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:38:54.449937 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:54.449915    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7ef57ed-1edc-4347-8369-5c9ab1748008-kube-api-access-flt7p" (OuterVolumeSpecName: "kube-api-access-flt7p") pod "b7ef57ed-1edc-4347-8369-5c9ab1748008" (UID: "b7ef57ed-1edc-4347-8369-5c9ab1748008"). InnerVolumeSpecName "kube-api-access-flt7p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:38:54.450118 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:54.449953    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7ef57ed-1edc-4347-8369-5c9ab1748008-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b7ef57ed-1edc-4347-8369-5c9ab1748008" (UID: "b7ef57ed-1edc-4347-8369-5c9ab1748008"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 05:38:54.549115 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:54.549076    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-flt7p\" (UniqueName: \"kubernetes.io/projected/b7ef57ed-1edc-4347-8369-5c9ab1748008-kube-api-access-flt7p\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:38:54.549115 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:54.549109    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7ef57ed-1edc-4347-8369-5c9ab1748008-openshift-service-ca\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:38:54.549115 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:54.549123    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7ef57ed-1edc-4347-8369-5c9ab1748008-maas-api-tls\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:38:54.752936 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:54.752860    2562 generic.go:356] "Generic (PLEG): container finished" podID="b7ef57ed-1edc-4347-8369-5c9ab1748008" containerID="1042a8d3a67a9d06c2bcd196e1b91c64da37566f2e563dbb28e0dfb13a27cc4d" exitCode=0
Jul 17 05:38:54.752936 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:54.752911    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-938bd0-78d6b65c78-8mqqx"
Jul 17 05:38:54.753133 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:54.752932    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-938bd0-78d6b65c78-8mqqx" event={"ID":"b7ef57ed-1edc-4347-8369-5c9ab1748008","Type":"ContainerDied","Data":"1042a8d3a67a9d06c2bcd196e1b91c64da37566f2e563dbb28e0dfb13a27cc4d"}
Jul 17 05:38:54.753133 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:54.753003    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-938bd0-78d6b65c78-8mqqx" event={"ID":"b7ef57ed-1edc-4347-8369-5c9ab1748008","Type":"ContainerDied","Data":"69c67ae27b6349f13a0184536aefba24f14fc672fcb59e2fdafd9c784d20423f"}
Jul 17 05:38:54.753133 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:54.753031    2562 scope.go:117] "RemoveContainer" containerID="1042a8d3a67a9d06c2bcd196e1b91c64da37566f2e563dbb28e0dfb13a27cc4d"
Jul 17 05:38:54.762327 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:54.762311    2562 scope.go:117] "RemoveContainer" containerID="1042a8d3a67a9d06c2bcd196e1b91c64da37566f2e563dbb28e0dfb13a27cc4d"
Jul 17 05:38:54.762599 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:38:54.762580    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1042a8d3a67a9d06c2bcd196e1b91c64da37566f2e563dbb28e0dfb13a27cc4d\": container with ID starting with 1042a8d3a67a9d06c2bcd196e1b91c64da37566f2e563dbb28e0dfb13a27cc4d not found: ID does not exist" containerID="1042a8d3a67a9d06c2bcd196e1b91c64da37566f2e563dbb28e0dfb13a27cc4d"
Jul 17 05:38:54.762665 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:54.762610    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1042a8d3a67a9d06c2bcd196e1b91c64da37566f2e563dbb28e0dfb13a27cc4d"} err="failed to get container status \"1042a8d3a67a9d06c2bcd196e1b91c64da37566f2e563dbb28e0dfb13a27cc4d\": rpc error: code = NotFound desc = could not find container \"1042a8d3a67a9d06c2bcd196e1b91c64da37566f2e563dbb28e0dfb13a27cc4d\": container with ID starting with 1042a8d3a67a9d06c2bcd196e1b91c64da37566f2e563dbb28e0dfb13a27cc4d not found: ID does not exist"
Jul 17 05:38:54.778350 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:54.778324    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-938bd0-78d6b65c78-8mqqx"]
Jul 17 05:38:54.784525 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:54.784507    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-938bd0-78d6b65c78-8mqqx"]
Jul 17 05:38:54.884902 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:38:54.884861    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7ef57ed-1edc-4347-8369-5c9ab1748008" path="/var/lib/kubelet/pods/b7ef57ed-1edc-4347-8369-5c9ab1748008/volumes"
Jul 17 05:39:09.052032 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:09.051997    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"]
Jul 17 05:39:09.052611 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:09.052212    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8" podUID="f4734a92-5798-4546-876e-96592d8964a4" containerName="istio-proxy" containerID="cri-o://eeb1ef4e59ecc568be79b8198177deb49f3ba303a1838b7f41b7068eb7dd9341" gracePeriod=30
Jul 17 05:39:11.493608 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:11.493569    2562 patch_prober.go:28] interesting pod/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 17 05:39:11.494148 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:11.493630    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8" podUID="f4734a92-5798-4546-876e-96592d8964a4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 17 05:39:14.298634 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:14.298609    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"
Jul 17 05:39:14.328008 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:14.327982    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4734a92-5798-4546-876e-96592d8964a4-credential-socket\") pod \"f4734a92-5798-4546-876e-96592d8964a4\" (UID: \"f4734a92-5798-4546-876e-96592d8964a4\") "
Jul 17 05:39:14.328175 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:14.328030    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2xvdz\" (UniqueName: \"kubernetes.io/projected/f4734a92-5798-4546-876e-96592d8964a4-kube-api-access-2xvdz\") pod \"f4734a92-5798-4546-876e-96592d8964a4\" (UID: \"f4734a92-5798-4546-876e-96592d8964a4\") "
Jul 17 05:39:14.328175 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:14.328058    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4734a92-5798-4546-876e-96592d8964a4-istio-podinfo\") pod \"f4734a92-5798-4546-876e-96592d8964a4\" (UID: \"f4734a92-5798-4546-876e-96592d8964a4\") "
Jul 17 05:39:14.328175 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:14.328092    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4734a92-5798-4546-876e-96592d8964a4-istio-token\") pod \"f4734a92-5798-4546-876e-96592d8964a4\" (UID: \"f4734a92-5798-4546-876e-96592d8964a4\") "
Jul 17 05:39:14.328175 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:14.328141    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4734a92-5798-4546-876e-96592d8964a4-istio-data\") pod \"f4734a92-5798-4546-876e-96592d8964a4\" (UID: \"f4734a92-5798-4546-876e-96592d8964a4\") "
Jul 17 05:39:14.328422 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:14.328196    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4734a92-5798-4546-876e-96592d8964a4-workload-certs\") pod \"f4734a92-5798-4546-876e-96592d8964a4\" (UID: \"f4734a92-5798-4546-876e-96592d8964a4\") "
Jul 17 05:39:14.328422 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:14.328232    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4734a92-5798-4546-876e-96592d8964a4-istiod-ca-cert\") pod \"f4734a92-5798-4546-876e-96592d8964a4\" (UID: \"f4734a92-5798-4546-876e-96592d8964a4\") "
Jul 17 05:39:14.328422 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:14.328265    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4734a92-5798-4546-876e-96592d8964a4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f4734a92-5798-4546-876e-96592d8964a4" (UID: "f4734a92-5798-4546-876e-96592d8964a4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:39:14.328422 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:14.328287    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4734a92-5798-4546-876e-96592d8964a4-istio-envoy\") pod \"f4734a92-5798-4546-876e-96592d8964a4\" (UID: \"f4734a92-5798-4546-876e-96592d8964a4\") "
Jul 17 05:39:14.328422 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:14.328355    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4734a92-5798-4546-876e-96592d8964a4-workload-socket\") pod \"f4734a92-5798-4546-876e-96592d8964a4\" (UID: \"f4734a92-5798-4546-876e-96592d8964a4\") "
Jul 17 05:39:14.328696 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:14.328471    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4734a92-5798-4546-876e-96592d8964a4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f4734a92-5798-4546-876e-96592d8964a4" (UID: "f4734a92-5798-4546-876e-96592d8964a4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:39:14.328696 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:14.328649    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4734a92-5798-4546-876e-96592d8964a4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f4734a92-5798-4546-876e-96592d8964a4" (UID: "f4734a92-5798-4546-876e-96592d8964a4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:39:14.328815 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:14.328699    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4734a92-5798-4546-876e-96592d8964a4-credential-socket\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:39:14.328815 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:14.328718    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4734a92-5798-4546-876e-96592d8964a4-workload-certs\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:39:14.328955 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:14.328922    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4734a92-5798-4546-876e-96592d8964a4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f4734a92-5798-4546-876e-96592d8964a4" (UID: "f4734a92-5798-4546-876e-96592d8964a4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:39:14.329307 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:14.329273    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4734a92-5798-4546-876e-96592d8964a4-istio-data" (OuterVolumeSpecName: "istio-data") pod "f4734a92-5798-4546-876e-96592d8964a4" (UID: "f4734a92-5798-4546-876e-96592d8964a4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:39:14.330515 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:14.330486    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f4734a92-5798-4546-876e-96592d8964a4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f4734a92-5798-4546-876e-96592d8964a4" (UID: "f4734a92-5798-4546-876e-96592d8964a4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 05:39:14.330623 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:14.330599    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4734a92-5798-4546-876e-96592d8964a4-istio-token" (OuterVolumeSpecName: "istio-token") pod "f4734a92-5798-4546-876e-96592d8964a4" (UID: "f4734a92-5798-4546-876e-96592d8964a4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:39:14.331091 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:14.331071    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4734a92-5798-4546-876e-96592d8964a4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f4734a92-5798-4546-876e-96592d8964a4" (UID: "f4734a92-5798-4546-876e-96592d8964a4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:39:14.331293 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:14.331270    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4734a92-5798-4546-876e-96592d8964a4-kube-api-access-2xvdz" (OuterVolumeSpecName: "kube-api-access-2xvdz") pod "f4734a92-5798-4546-876e-96592d8964a4" (UID: "f4734a92-5798-4546-876e-96592d8964a4"). InnerVolumeSpecName "kube-api-access-2xvdz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:39:14.429795 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:14.429764    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4734a92-5798-4546-876e-96592d8964a4-istiod-ca-cert\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:39:14.429795 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:14.429789    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4734a92-5798-4546-876e-96592d8964a4-istio-envoy\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:39:14.429795 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:14.429798    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4734a92-5798-4546-876e-96592d8964a4-workload-socket\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:39:14.430037 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:14.429806    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2xvdz\" (UniqueName: \"kubernetes.io/projected/f4734a92-5798-4546-876e-96592d8964a4-kube-api-access-2xvdz\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:39:14.430037 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:14.429815    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4734a92-5798-4546-876e-96592d8964a4-istio-podinfo\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:39:14.430037 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:14.429824    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4734a92-5798-4546-876e-96592d8964a4-istio-token\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:39:14.430037 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:14.429831    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4734a92-5798-4546-876e-96592d8964a4-istio-data\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:39:14.859620 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:14.859588    2562 generic.go:356] "Generic (PLEG): container finished" podID="f4734a92-5798-4546-876e-96592d8964a4" containerID="eeb1ef4e59ecc568be79b8198177deb49f3ba303a1838b7f41b7068eb7dd9341" exitCode=0
Jul 17 05:39:14.859772 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:14.859636    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"
Jul 17 05:39:14.859772 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:14.859675    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8" event={"ID":"f4734a92-5798-4546-876e-96592d8964a4","Type":"ContainerDied","Data":"eeb1ef4e59ecc568be79b8198177deb49f3ba303a1838b7f41b7068eb7dd9341"}
Jul 17 05:39:14.859772 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:14.859732    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8" event={"ID":"f4734a92-5798-4546-876e-96592d8964a4","Type":"ContainerDied","Data":"f81fc2adccf8ed58a4c8f28c9e14f9d96dfc754ec12f4b52e1d4dfda2fe220a5"}
Jul 17 05:39:14.859772 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:14.859756    2562 scope.go:117] "RemoveContainer" containerID="eeb1ef4e59ecc568be79b8198177deb49f3ba303a1838b7f41b7068eb7dd9341"
Jul 17 05:39:14.869206 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:14.869188    2562 scope.go:117] "RemoveContainer" containerID="cb1c9062729f7e87e08c8b7209c23521e628222f0c29a4dbd630f6344223e97d"
Jul 17 05:39:14.877299 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:14.877281    2562 scope.go:117] "RemoveContainer" containerID="eeb1ef4e59ecc568be79b8198177deb49f3ba303a1838b7f41b7068eb7dd9341"
Jul 17 05:39:14.877537 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:39:14.877517    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eeb1ef4e59ecc568be79b8198177deb49f3ba303a1838b7f41b7068eb7dd9341\": container with ID starting with eeb1ef4e59ecc568be79b8198177deb49f3ba303a1838b7f41b7068eb7dd9341 not found: ID does not exist" containerID="eeb1ef4e59ecc568be79b8198177deb49f3ba303a1838b7f41b7068eb7dd9341"
Jul 17 05:39:14.877619 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:14.877548    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eeb1ef4e59ecc568be79b8198177deb49f3ba303a1838b7f41b7068eb7dd9341"} err="failed to get container status \"eeb1ef4e59ecc568be79b8198177deb49f3ba303a1838b7f41b7068eb7dd9341\": rpc error: code = NotFound desc = could not find container \"eeb1ef4e59ecc568be79b8198177deb49f3ba303a1838b7f41b7068eb7dd9341\": container with ID starting with eeb1ef4e59ecc568be79b8198177deb49f3ba303a1838b7f41b7068eb7dd9341 not found: ID does not exist"
Jul 17 05:39:14.877619 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:14.877572    2562 scope.go:117] "RemoveContainer" containerID="cb1c9062729f7e87e08c8b7209c23521e628222f0c29a4dbd630f6344223e97d"
Jul 17 05:39:14.877832 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:39:14.877812    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb1c9062729f7e87e08c8b7209c23521e628222f0c29a4dbd630f6344223e97d\": container with ID starting with cb1c9062729f7e87e08c8b7209c23521e628222f0c29a4dbd630f6344223e97d not found: ID does not exist" containerID="cb1c9062729f7e87e08c8b7209c23521e628222f0c29a4dbd630f6344223e97d"
Jul 17 05:39:14.877887 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:14.877841    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb1c9062729f7e87e08c8b7209c23521e628222f0c29a4dbd630f6344223e97d"} err="failed to get container status \"cb1c9062729f7e87e08c8b7209c23521e628222f0c29a4dbd630f6344223e97d\": rpc error: code = NotFound desc = could not find container \"cb1c9062729f7e87e08c8b7209c23521e628222f0c29a4dbd630f6344223e97d\": container with ID starting with cb1c9062729f7e87e08c8b7209c23521e628222f0c29a4dbd630f6344223e97d not found: ID does not exist"
Jul 17 05:39:14.884711 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:14.884688    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"]
Jul 17 05:39:14.885854 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:14.885834    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-938bd0-openshift-default-7dd69594c-wzjm8"]
Jul 17 05:39:16.884360 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:16.884327    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4734a92-5798-4546-876e-96592d8964a4" path="/var/lib/kubelet/pods/f4734a92-5798-4546-876e-96592d8964a4/volumes"
Jul 17 05:39:35.288141 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:35.288049    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s"]
Jul 17 05:39:35.288620 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:35.288315    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s" podUID="7190c888-53d0-428e-901a-f8d87456541c" containerName="istio-proxy" containerID="cri-o://55657eeb5605df305bfb6a80ee75bc05b784696c905088134fbdad244e290401" gracePeriod=30
Jul 17 05:39:36.222421 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.222380    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6"]
Jul 17 05:39:36.222908 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.222888    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4734a92-5798-4546-876e-96592d8964a4" containerName="istio-proxy"
Jul 17 05:39:36.223049 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.222909    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4734a92-5798-4546-876e-96592d8964a4" containerName="istio-proxy"
Jul 17 05:39:36.223049 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.222933    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7ef57ed-1edc-4347-8369-5c9ab1748008" containerName="maas-api"
Jul 17 05:39:36.223049 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.222943    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7ef57ed-1edc-4347-8369-5c9ab1748008" containerName="maas-api"
Jul 17 05:39:36.223049 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.222956    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4734a92-5798-4546-876e-96592d8964a4" containerName="istio-proxy"
Jul 17 05:39:36.223049 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.222984    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4734a92-5798-4546-876e-96592d8964a4" containerName="istio-proxy"
Jul 17 05:39:36.223312 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.223114    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4734a92-5798-4546-876e-96592d8964a4" containerName="istio-proxy"
Jul 17 05:39:36.223312 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.223126    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7ef57ed-1edc-4347-8369-5c9ab1748008" containerName="maas-api"
Jul 17 05:39:36.225446 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.225425    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6"
Jul 17 05:39:36.228355 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.228335    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-560f5b-openshift-default-dockercfg-jr7t5\""
Jul 17 05:39:36.238507 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.238482    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6"]
Jul 17 05:39:36.333668 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.333629    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0357b315-a3ca-4c24-b0e5-51ee48d89983-istiod-ca-cert\") pod \"e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6\" (UID: \"0357b315-a3ca-4c24-b0e5-51ee48d89983\") " pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6"
Jul 17 05:39:36.334092 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.333676    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0357b315-a3ca-4c24-b0e5-51ee48d89983-istio-envoy\") pod \"e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6\" (UID: \"0357b315-a3ca-4c24-b0e5-51ee48d89983\") " pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6"
Jul 17 05:39:36.334092 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.333711    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0357b315-a3ca-4c24-b0e5-51ee48d89983-credential-socket\") pod \"e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6\" (UID: \"0357b315-a3ca-4c24-b0e5-51ee48d89983\") " pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6"
Jul 17 05:39:36.334092 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.333738    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0357b315-a3ca-4c24-b0e5-51ee48d89983-istio-token\") pod \"e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6\" (UID: \"0357b315-a3ca-4c24-b0e5-51ee48d89983\") " pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6"
Jul 17 05:39:36.334092 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.333774    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0357b315-a3ca-4c24-b0e5-51ee48d89983-workload-socket\") pod \"e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6\" (UID: \"0357b315-a3ca-4c24-b0e5-51ee48d89983\") " pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6"
Jul 17 05:39:36.334092 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.333853    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0357b315-a3ca-4c24-b0e5-51ee48d89983-istio-data\") pod \"e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6\" (UID: \"0357b315-a3ca-4c24-b0e5-51ee48d89983\") " pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6"
Jul 17 05:39:36.334092 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.333903    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0357b315-a3ca-4c24-b0e5-51ee48d89983-istio-podinfo\") pod \"e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6\" (UID: \"0357b315-a3ca-4c24-b0e5-51ee48d89983\") " pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6"
Jul 17 05:39:36.334092 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.333924    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qwgw\" (UniqueName: \"kubernetes.io/projected/0357b315-a3ca-4c24-b0e5-51ee48d89983-kube-api-access-8qwgw\") pod \"e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6\" (UID: \"0357b315-a3ca-4c24-b0e5-51ee48d89983\") " pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6"
Jul 17 05:39:36.334092 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.333996    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0357b315-a3ca-4c24-b0e5-51ee48d89983-workload-certs\") pod \"e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6\" (UID: \"0357b315-a3ca-4c24-b0e5-51ee48d89983\") " pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6"
Jul 17 05:39:36.435082 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.435049    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0357b315-a3ca-4c24-b0e5-51ee48d89983-workload-socket\") pod \"e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6\" (UID: \"0357b315-a3ca-4c24-b0e5-51ee48d89983\") " pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6"
Jul 17 05:39:36.435266 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.435095    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0357b315-a3ca-4c24-b0e5-51ee48d89983-istio-data\") pod \"e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6\" (UID: \"0357b315-a3ca-4c24-b0e5-51ee48d89983\") " pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6"
Jul 17 05:39:36.435266 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.435121    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0357b315-a3ca-4c24-b0e5-51ee48d89983-istio-podinfo\") pod \"e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6\" (UID: \"0357b315-a3ca-4c24-b0e5-51ee48d89983\") " pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6"
Jul 17 05:39:36.435266 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.435144    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qwgw\" (UniqueName: \"kubernetes.io/projected/0357b315-a3ca-4c24-b0e5-51ee48d89983-kube-api-access-8qwgw\") pod \"e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6\" (UID: \"0357b315-a3ca-4c24-b0e5-51ee48d89983\") " pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6"
Jul 17 05:39:36.435266 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.435204    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0357b315-a3ca-4c24-b0e5-51ee48d89983-workload-certs\") pod \"e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6\" (UID: \"0357b315-a3ca-4c24-b0e5-51ee48d89983\") " pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6"
Jul 17 05:39:36.435266 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.435252    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0357b315-a3ca-4c24-b0e5-51ee48d89983-istiod-ca-cert\") pod \"e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6\" (UID: \"0357b315-a3ca-4c24-b0e5-51ee48d89983\") " pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6"
Jul 17 05:39:36.435522 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.435284    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0357b315-a3ca-4c24-b0e5-51ee48d89983-istio-envoy\") pod \"e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6\" (UID: \"0357b315-a3ca-4c24-b0e5-51ee48d89983\") " pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6"
Jul 17 05:39:36.435522 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.435320    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0357b315-a3ca-4c24-b0e5-51ee48d89983-credential-socket\") pod \"e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6\" (UID: \"0357b315-a3ca-4c24-b0e5-51ee48d89983\") " pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6"
Jul 17 05:39:36.435522 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.435352    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0357b315-a3ca-4c24-b0e5-51ee48d89983-istio-token\") pod \"e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6\" (UID: \"0357b315-a3ca-4c24-b0e5-51ee48d89983\") " pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6"
Jul 17 05:39:36.435522 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.435503    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0357b315-a3ca-4c24-b0e5-51ee48d89983-istio-data\") pod \"e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6\" (UID: \"0357b315-a3ca-4c24-b0e5-51ee48d89983\") " pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6"
Jul 17 05:39:36.435723 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.435551    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0357b315-a3ca-4c24-b0e5-51ee48d89983-workload-socket\") pod \"e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6\" (UID: \"0357b315-a3ca-4c24-b0e5-51ee48d89983\") " pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6"
Jul 17 05:39:36.435723 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.435572    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0357b315-a3ca-4c24-b0e5-51ee48d89983-workload-certs\") pod \"e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6\" (UID: \"0357b315-a3ca-4c24-b0e5-51ee48d89983\") " pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6"
Jul 17 05:39:36.435824 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.435787    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0357b315-a3ca-4c24-b0e5-51ee48d89983-credential-socket\") pod \"e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6\" (UID: \"0357b315-a3ca-4c24-b0e5-51ee48d89983\") " pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6"
Jul 17 05:39:36.436024 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.436005    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0357b315-a3ca-4c24-b0e5-51ee48d89983-istiod-ca-cert\") pod \"e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6\" (UID: \"0357b315-a3ca-4c24-b0e5-51ee48d89983\") " pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6"
Jul 17 05:39:36.437691 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.437669    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0357b315-a3ca-4c24-b0e5-51ee48d89983-istio-envoy\") pod \"e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6\" (UID: \"0357b315-a3ca-4c24-b0e5-51ee48d89983\") " pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6"
Jul 17 05:39:36.437828 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.437809    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0357b315-a3ca-4c24-b0e5-51ee48d89983-istio-podinfo\") pod \"e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6\" (UID: \"0357b315-a3ca-4c24-b0e5-51ee48d89983\") " pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6"
Jul 17 05:39:36.444423 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.444396    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0357b315-a3ca-4c24-b0e5-51ee48d89983-istio-token\") pod \"e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6\" (UID: \"0357b315-a3ca-4c24-b0e5-51ee48d89983\") " pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6"
Jul 17 05:39:36.444555 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.444522    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qwgw\" (UniqueName: \"kubernetes.io/projected/0357b315-a3ca-4c24-b0e5-51ee48d89983-kube-api-access-8qwgw\") pod \"e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6\" (UID: \"0357b315-a3ca-4c24-b0e5-51ee48d89983\") " pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6"
Jul 17 05:39:36.536743 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.536657    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6"
Jul 17 05:39:36.678511 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.678480    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6"]
Jul 17 05:39:36.680618 ip-10-0-132-19 kubenswrapper[2562]: W0717 05:39:36.680591    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0357b315_a3ca_4c24_b0e5_51ee48d89983.slice/crio-44b5a9ca2352f7953469bfbd163a0cd45e5a3c93838759e4b69d59b45be79af0 WatchSource:0}: Error finding container 44b5a9ca2352f7953469bfbd163a0cd45e5a3c93838759e4b69d59b45be79af0: Status 404 returned error can't find the container with id 44b5a9ca2352f7953469bfbd163a0cd45e5a3c93838759e4b69d59b45be79af0
Jul 17 05:39:36.682746 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.682716    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:39:36.682852 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.682824    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:39:36.682915 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.682871    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:39:36.975866 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.975826    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6" event={"ID":"0357b315-a3ca-4c24-b0e5-51ee48d89983","Type":"ContainerStarted","Data":"4d76cd160f76415e079135eda1f1309d69825c2099a7df0deb78d976f46560fd"}
Jul 17 05:39:36.976099 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.975883    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6" event={"ID":"0357b315-a3ca-4c24-b0e5-51ee48d89983","Type":"ContainerStarted","Data":"44b5a9ca2352f7953469bfbd163a0cd45e5a3c93838759e4b69d59b45be79af0"}
Jul 17 05:39:36.999326 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:36.999273    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6" podStartSLOduration=0.99925361 podStartE2EDuration="999.25361ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:39:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:39:36.998145 +0000 UTC m=+2964.681738685" watchObservedRunningTime="2026-07-17 05:39:36.99925361 +0000 UTC m=+2964.682847266"
Jul 17 05:39:37.144356 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:37.144310    2562 patch_prober.go:28] interesting pod/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Jul 17 05:39:37.144534 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:37.144391    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s" podUID="7190c888-53d0-428e-901a-f8d87456541c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 17 05:39:37.536792 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:37.536759    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6"
Jul 17 05:39:37.541663 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:37.541642    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6"
Jul 17 05:39:37.982017 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:37.981987    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6"
Jul 17 05:39:37.983047 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:37.983027    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6"
Jul 17 05:39:40.544021 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:40.543990    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s"
Jul 17 05:39:40.676411 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:40.676379    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7190c888-53d0-428e-901a-f8d87456541c-workload-certs\") pod \"7190c888-53d0-428e-901a-f8d87456541c\" (UID: \"7190c888-53d0-428e-901a-f8d87456541c\") "
Jul 17 05:39:40.676411 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:40.676417    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7190c888-53d0-428e-901a-f8d87456541c-istio-token\") pod \"7190c888-53d0-428e-901a-f8d87456541c\" (UID: \"7190c888-53d0-428e-901a-f8d87456541c\") "
Jul 17 05:39:40.676662 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:40.676436    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7190c888-53d0-428e-901a-f8d87456541c-credential-socket\") pod \"7190c888-53d0-428e-901a-f8d87456541c\" (UID: \"7190c888-53d0-428e-901a-f8d87456541c\") "
Jul 17 05:39:40.676662 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:40.676455    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7190c888-53d0-428e-901a-f8d87456541c-istiod-ca-cert\") pod \"7190c888-53d0-428e-901a-f8d87456541c\" (UID: \"7190c888-53d0-428e-901a-f8d87456541c\") "
Jul 17 05:39:40.676662 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:40.676486    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7190c888-53d0-428e-901a-f8d87456541c-istio-data\") pod \"7190c888-53d0-428e-901a-f8d87456541c\" (UID: \"7190c888-53d0-428e-901a-f8d87456541c\") "
Jul 17 05:39:40.676662 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:40.676602    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7190c888-53d0-428e-901a-f8d87456541c-istio-envoy\") pod \"7190c888-53d0-428e-901a-f8d87456541c\" (UID: \"7190c888-53d0-428e-901a-f8d87456541c\") "
Jul 17 05:39:40.676662 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:40.676645    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7190c888-53d0-428e-901a-f8d87456541c-workload-socket\") pod \"7190c888-53d0-428e-901a-f8d87456541c\" (UID: \"7190c888-53d0-428e-901a-f8d87456541c\") "
Jul 17 05:39:40.676920 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:40.676724    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7190c888-53d0-428e-901a-f8d87456541c-istio-podinfo\") pod \"7190c888-53d0-428e-901a-f8d87456541c\" (UID: \"7190c888-53d0-428e-901a-f8d87456541c\") "
Jul 17 05:39:40.676920 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:40.676779    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-skrs9\" (UniqueName: \"kubernetes.io/projected/7190c888-53d0-428e-901a-f8d87456541c-kube-api-access-skrs9\") pod \"7190c888-53d0-428e-901a-f8d87456541c\" (UID: \"7190c888-53d0-428e-901a-f8d87456541c\") "
Jul 17 05:39:40.676920 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:40.676801    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7190c888-53d0-428e-901a-f8d87456541c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7190c888-53d0-428e-901a-f8d87456541c" (UID: "7190c888-53d0-428e-901a-f8d87456541c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:39:40.676920 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:40.676821    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7190c888-53d0-428e-901a-f8d87456541c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7190c888-53d0-428e-901a-f8d87456541c" (UID: "7190c888-53d0-428e-901a-f8d87456541c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:39:40.676920 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:40.676902    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7190c888-53d0-428e-901a-f8d87456541c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7190c888-53d0-428e-901a-f8d87456541c" (UID: "7190c888-53d0-428e-901a-f8d87456541c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:39:40.677218 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:40.677192    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7190c888-53d0-428e-901a-f8d87456541c-workload-certs\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:39:40.677218 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:40.677213    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7190c888-53d0-428e-901a-f8d87456541c-credential-socket\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:39:40.677502 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:40.677227    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7190c888-53d0-428e-901a-f8d87456541c-workload-socket\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:39:40.677502 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:40.677339    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7190c888-53d0-428e-901a-f8d87456541c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7190c888-53d0-428e-901a-f8d87456541c" (UID: "7190c888-53d0-428e-901a-f8d87456541c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:39:40.677606 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:40.677575    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7190c888-53d0-428e-901a-f8d87456541c-istio-data" (OuterVolumeSpecName: "istio-data") pod "7190c888-53d0-428e-901a-f8d87456541c" (UID: "7190c888-53d0-428e-901a-f8d87456541c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:39:40.679080 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:40.679046    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7190c888-53d0-428e-901a-f8d87456541c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7190c888-53d0-428e-901a-f8d87456541c" (UID: "7190c888-53d0-428e-901a-f8d87456541c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:39:40.679206 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:40.679088    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7190c888-53d0-428e-901a-f8d87456541c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7190c888-53d0-428e-901a-f8d87456541c" (UID: "7190c888-53d0-428e-901a-f8d87456541c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 05:39:40.679330 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:40.679302    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7190c888-53d0-428e-901a-f8d87456541c-istio-token" (OuterVolumeSpecName: "istio-token") pod "7190c888-53d0-428e-901a-f8d87456541c" (UID: "7190c888-53d0-428e-901a-f8d87456541c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:39:40.679399 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:40.679352    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7190c888-53d0-428e-901a-f8d87456541c-kube-api-access-skrs9" (OuterVolumeSpecName: "kube-api-access-skrs9") pod "7190c888-53d0-428e-901a-f8d87456541c" (UID: "7190c888-53d0-428e-901a-f8d87456541c"). InnerVolumeSpecName "kube-api-access-skrs9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:39:40.778268 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:40.778227    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7190c888-53d0-428e-901a-f8d87456541c-istio-token\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:39:40.778268 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:40.778259    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7190c888-53d0-428e-901a-f8d87456541c-istiod-ca-cert\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:39:40.778268 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:40.778267    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7190c888-53d0-428e-901a-f8d87456541c-istio-data\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:39:40.778268 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:40.778275    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7190c888-53d0-428e-901a-f8d87456541c-istio-envoy\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:39:40.778540 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:40.778283    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7190c888-53d0-428e-901a-f8d87456541c-istio-podinfo\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:39:40.778540 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:40.778291    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-skrs9\" (UniqueName: \"kubernetes.io/projected/7190c888-53d0-428e-901a-f8d87456541c-kube-api-access-skrs9\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:39:40.999680 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:40.999578    2562 generic.go:356] "Generic (PLEG): container finished" podID="7190c888-53d0-428e-901a-f8d87456541c" containerID="55657eeb5605df305bfb6a80ee75bc05b784696c905088134fbdad244e290401" exitCode=0
Jul 17 05:39:40.999680 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:40.999635    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s"
Jul 17 05:39:40.999680 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:40.999659    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s" event={"ID":"7190c888-53d0-428e-901a-f8d87456541c","Type":"ContainerDied","Data":"55657eeb5605df305bfb6a80ee75bc05b784696c905088134fbdad244e290401"}
Jul 17 05:39:40.999935 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:40.999708    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s" event={"ID":"7190c888-53d0-428e-901a-f8d87456541c","Type":"ContainerDied","Data":"cd6e18d05861891cbad574e586bb47609a10a5f76846eb8e2078d3ce5d70361b"}
Jul 17 05:39:40.999935 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:40.999735    2562 scope.go:117] "RemoveContainer" containerID="55657eeb5605df305bfb6a80ee75bc05b784696c905088134fbdad244e290401"
Jul 17 05:39:41.008781 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:41.008765    2562 scope.go:117] "RemoveContainer" containerID="6ee7411a8ed8c84ea2633f944e1c18620b2ac2ece470c23ea13719b22ea4643c"
Jul 17 05:39:41.017513 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:41.017493    2562 scope.go:117] "RemoveContainer" containerID="55657eeb5605df305bfb6a80ee75bc05b784696c905088134fbdad244e290401"
Jul 17 05:39:41.017757 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:39:41.017739    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55657eeb5605df305bfb6a80ee75bc05b784696c905088134fbdad244e290401\": container with ID starting with 55657eeb5605df305bfb6a80ee75bc05b784696c905088134fbdad244e290401 not found: ID does not exist" containerID="55657eeb5605df305bfb6a80ee75bc05b784696c905088134fbdad244e290401"
Jul 17 05:39:41.017832 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:41.017764    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55657eeb5605df305bfb6a80ee75bc05b784696c905088134fbdad244e290401"} err="failed to get container status \"55657eeb5605df305bfb6a80ee75bc05b784696c905088134fbdad244e290401\": rpc error: code = NotFound desc = could not find container \"55657eeb5605df305bfb6a80ee75bc05b784696c905088134fbdad244e290401\": container with ID starting with 55657eeb5605df305bfb6a80ee75bc05b784696c905088134fbdad244e290401 not found: ID does not exist"
Jul 17 05:39:41.017832 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:41.017780    2562 scope.go:117] "RemoveContainer" containerID="6ee7411a8ed8c84ea2633f944e1c18620b2ac2ece470c23ea13719b22ea4643c"
Jul 17 05:39:41.018012 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:39:41.017994    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ee7411a8ed8c84ea2633f944e1c18620b2ac2ece470c23ea13719b22ea4643c\": container with ID starting with 6ee7411a8ed8c84ea2633f944e1c18620b2ac2ece470c23ea13719b22ea4643c not found: ID does not exist" containerID="6ee7411a8ed8c84ea2633f944e1c18620b2ac2ece470c23ea13719b22ea4643c"
Jul 17 05:39:41.018094 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:41.018019    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ee7411a8ed8c84ea2633f944e1c18620b2ac2ece470c23ea13719b22ea4643c"} err="failed to get container status \"6ee7411a8ed8c84ea2633f944e1c18620b2ac2ece470c23ea13719b22ea4643c\": rpc error: code = NotFound desc = could not find container \"6ee7411a8ed8c84ea2633f944e1c18620b2ac2ece470c23ea13719b22ea4643c\": container with ID starting with 6ee7411a8ed8c84ea2633f944e1c18620b2ac2ece470c23ea13719b22ea4643c not found: ID does not exist"
Jul 17 05:39:41.019214 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:41.019193    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s"]
Jul 17 05:39:41.022442 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:41.022423    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-ce2966-openshift-default-5cc67ddcdc-lsg8s"]
Jul 17 05:39:42.887772 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:42.887740    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7190c888-53d0-428e-901a-f8d87456541c" path="/var/lib/kubelet/pods/7190c888-53d0-428e-901a-f8d87456541c/volumes"
Jul 17 05:39:44.098707 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:44.098663    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx"]
Jul 17 05:39:44.099420 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:44.099401    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7190c888-53d0-428e-901a-f8d87456541c" containerName="istio-proxy"
Jul 17 05:39:44.099488 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:44.099426    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7190c888-53d0-428e-901a-f8d87456541c" containerName="istio-proxy"
Jul 17 05:39:44.099488 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:44.099438    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7190c888-53d0-428e-901a-f8d87456541c" containerName="istio-proxy"
Jul 17 05:39:44.099488 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:44.099443    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7190c888-53d0-428e-901a-f8d87456541c" containerName="istio-proxy"
Jul 17 05:39:44.099587 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:44.099514    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4734a92-5798-4546-876e-96592d8964a4" containerName="istio-proxy"
Jul 17 05:39:44.099587 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:44.099522    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7190c888-53d0-428e-901a-f8d87456541c" containerName="istio-proxy"
Jul 17 05:39:44.099587 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:44.099528    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7190c888-53d0-428e-901a-f8d87456541c" containerName="istio-proxy"
Jul 17 05:39:44.102483 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:44.102464    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx"
Jul 17 05:39:44.106526 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:44.106505    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-823cda-openshift-default-dockercfg-shfwl\""
Jul 17 05:39:44.117742 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:44.117713    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx"]
Jul 17 05:39:44.205876 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:44.205839    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99aa5e8d-f861-41d8-a8db-e764ae921284-workload-certs\") pod \"e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx\" (UID: \"99aa5e8d-f861-41d8-a8db-e764ae921284\") " pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx"
Jul 17 05:39:44.206114 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:44.205884    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99aa5e8d-f861-41d8-a8db-e764ae921284-credential-socket\") pod \"e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx\" (UID: \"99aa5e8d-f861-41d8-a8db-e764ae921284\") " pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx"
Jul 17 05:39:44.206114 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:44.206016    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99aa5e8d-f861-41d8-a8db-e764ae921284-istio-envoy\") pod \"e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx\" (UID: \"99aa5e8d-f861-41d8-a8db-e764ae921284\") " pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx"
Jul 17 05:39:44.206114 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:44.206057    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99aa5e8d-f861-41d8-a8db-e764ae921284-istio-podinfo\") pod \"e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx\" (UID: \"99aa5e8d-f861-41d8-a8db-e764ae921284\") " pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx"
Jul 17 05:39:44.206302 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:44.206121    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99aa5e8d-f861-41d8-a8db-e764ae921284-istiod-ca-cert\") pod \"e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx\" (UID: \"99aa5e8d-f861-41d8-a8db-e764ae921284\") " pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx"
Jul 17 05:39:44.206302 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:44.206173    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99aa5e8d-f861-41d8-a8db-e764ae921284-istio-data\") pod \"e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx\" (UID: \"99aa5e8d-f861-41d8-a8db-e764ae921284\") " pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx"
Jul 17 05:39:44.206302 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:44.206225    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99aa5e8d-f861-41d8-a8db-e764ae921284-istio-token\") pod \"e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx\" (UID: \"99aa5e8d-f861-41d8-a8db-e764ae921284\") " pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx"
Jul 17 05:39:44.206302 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:44.206247    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99aa5e8d-f861-41d8-a8db-e764ae921284-workload-socket\") pod \"e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx\" (UID: \"99aa5e8d-f861-41d8-a8db-e764ae921284\") " pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx"
Jul 17 05:39:44.206457 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:44.206314    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lwmw\" (UniqueName: \"kubernetes.io/projected/99aa5e8d-f861-41d8-a8db-e764ae921284-kube-api-access-9lwmw\") pod \"e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx\" (UID: \"99aa5e8d-f861-41d8-a8db-e764ae921284\") " pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx"
Jul 17 05:39:44.306827 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:44.306789    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99aa5e8d-f861-41d8-a8db-e764ae921284-istio-envoy\") pod \"e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx\" (UID: \"99aa5e8d-f861-41d8-a8db-e764ae921284\") " pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx"
Jul 17 05:39:44.307062 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:44.306836    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99aa5e8d-f861-41d8-a8db-e764ae921284-istio-podinfo\") pod \"e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx\" (UID: \"99aa5e8d-f861-41d8-a8db-e764ae921284\") " pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx"
Jul 17 05:39:44.307062 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:44.306859    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99aa5e8d-f861-41d8-a8db-e764ae921284-istiod-ca-cert\") pod \"e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx\" (UID: \"99aa5e8d-f861-41d8-a8db-e764ae921284\") " pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx"
Jul 17 05:39:44.307062 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:44.306887    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99aa5e8d-f861-41d8-a8db-e764ae921284-istio-data\") pod \"e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx\" (UID: \"99aa5e8d-f861-41d8-a8db-e764ae921284\") " pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx"
Jul 17 05:39:44.307062 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:44.306928    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99aa5e8d-f861-41d8-a8db-e764ae921284-istio-token\") pod \"e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx\" (UID: \"99aa5e8d-f861-41d8-a8db-e764ae921284\") " pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx"
Jul 17 05:39:44.307062 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:44.306948    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99aa5e8d-f861-41d8-a8db-e764ae921284-workload-socket\") pod \"e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx\" (UID: \"99aa5e8d-f861-41d8-a8db-e764ae921284\") " pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx"
Jul 17 05:39:44.307062 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:44.307009    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lwmw\" (UniqueName: \"kubernetes.io/projected/99aa5e8d-f861-41d8-a8db-e764ae921284-kube-api-access-9lwmw\") pod \"e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx\" (UID: \"99aa5e8d-f861-41d8-a8db-e764ae921284\") " pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx"
Jul 17 05:39:44.307062 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:44.307047    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99aa5e8d-f861-41d8-a8db-e764ae921284-workload-certs\") pod \"e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx\" (UID: \"99aa5e8d-f861-41d8-a8db-e764ae921284\") " pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx"
Jul 17 05:39:44.307437 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:44.307071    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99aa5e8d-f861-41d8-a8db-e764ae921284-credential-socket\") pod \"e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx\" (UID: \"99aa5e8d-f861-41d8-a8db-e764ae921284\") " pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx"
Jul 17 05:39:44.307437 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:44.307386    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99aa5e8d-f861-41d8-a8db-e764ae921284-istio-data\") pod \"e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx\" (UID: \"99aa5e8d-f861-41d8-a8db-e764ae921284\") " pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx"
Jul 17 05:39:44.307555 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:44.307470    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99aa5e8d-f861-41d8-a8db-e764ae921284-workload-socket\") pod \"e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx\" (UID: \"99aa5e8d-f861-41d8-a8db-e764ae921284\") " pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx"
Jul 17 05:39:44.307610 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:44.307553    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99aa5e8d-f861-41d8-a8db-e764ae921284-workload-certs\") pod \"e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx\" (UID: \"99aa5e8d-f861-41d8-a8db-e764ae921284\") " pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx"
Jul 17 05:39:44.307650 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:44.307609    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99aa5e8d-f861-41d8-a8db-e764ae921284-credential-socket\") pod \"e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx\" (UID: \"99aa5e8d-f861-41d8-a8db-e764ae921284\") " pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx"
Jul 17 05:39:44.307901 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:44.307880    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99aa5e8d-f861-41d8-a8db-e764ae921284-istiod-ca-cert\") pod \"e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx\" (UID: \"99aa5e8d-f861-41d8-a8db-e764ae921284\") " pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx"
Jul 17 05:39:44.309247 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:44.309227    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99aa5e8d-f861-41d8-a8db-e764ae921284-istio-envoy\") pod \"e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx\" (UID: \"99aa5e8d-f861-41d8-a8db-e764ae921284\") " pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx"
Jul 17 05:39:44.309446 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:44.309429    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99aa5e8d-f861-41d8-a8db-e764ae921284-istio-podinfo\") pod \"e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx\" (UID: \"99aa5e8d-f861-41d8-a8db-e764ae921284\") " pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx"
Jul 17 05:39:44.315789 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:44.315759    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99aa5e8d-f861-41d8-a8db-e764ae921284-istio-token\") pod \"e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx\" (UID: \"99aa5e8d-f861-41d8-a8db-e764ae921284\") " pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx"
Jul 17 05:39:44.315889 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:44.315878    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lwmw\" (UniqueName: \"kubernetes.io/projected/99aa5e8d-f861-41d8-a8db-e764ae921284-kube-api-access-9lwmw\") pod \"e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx\" (UID: \"99aa5e8d-f861-41d8-a8db-e764ae921284\") " pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx"
Jul 17 05:39:44.413645 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:44.413221    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx"
Jul 17 05:39:44.547354 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:44.547323    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx"]
Jul 17 05:39:44.549486 ip-10-0-132-19 kubenswrapper[2562]: W0717 05:39:44.549462    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99aa5e8d_f861_41d8_a8db_e764ae921284.slice/crio-49f2eaeef79b2d72534234702ea0bbfc308743e6ef3453532a4a8cc6ae5a1beb WatchSource:0}: Error finding container 49f2eaeef79b2d72534234702ea0bbfc308743e6ef3453532a4a8cc6ae5a1beb: Status 404 returned error can't find the container with id 49f2eaeef79b2d72534234702ea0bbfc308743e6ef3453532a4a8cc6ae5a1beb
Jul 17 05:39:44.551686 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:44.551654    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:39:44.551765 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:44.551719    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:39:44.551765 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:44.551748    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:39:45.022343 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:45.022309    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx" event={"ID":"99aa5e8d-f861-41d8-a8db-e764ae921284","Type":"ContainerStarted","Data":"ebc95b438fd814482b1975dc9ffdb71570d5dda8ee21bc833cedaf8c646ba3f0"}
Jul 17 05:39:45.022562 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:45.022361    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx" event={"ID":"99aa5e8d-f861-41d8-a8db-e764ae921284","Type":"ContainerStarted","Data":"49f2eaeef79b2d72534234702ea0bbfc308743e6ef3453532a4a8cc6ae5a1beb"}
Jul 17 05:39:45.047693 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:45.047643    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx" podStartSLOduration=1.047623358 podStartE2EDuration="1.047623358s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:39:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:39:45.046039712 +0000 UTC m=+2972.729633390" watchObservedRunningTime="2026-07-17 05:39:45.047623358 +0000 UTC m=+2972.731217022"
Jul 17 05:39:45.413863 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:45.413826    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx"
Jul 17 05:39:45.418923 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:45.418896    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx"
Jul 17 05:39:46.027379 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:46.027352    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx"
Jul 17 05:39:46.028412 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:46.028391    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx"
Jul 17 05:39:52.998149 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:52.997685    2562 patch_prober.go:28] interesting pod/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": read tcp 10.134.0.2:38508->10.134.0.69:15021: read: connection reset by peer" start-of-body=
Jul 17 05:39:52.998149 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:52.997753    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6" podUID="0357b315-a3ca-4c24-b0e5-51ee48d89983" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": read tcp 10.134.0.2:38508->10.134.0.69:15021: read: connection reset by peer"
Jul 17 05:39:53.079154 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:53.079111    2562 generic.go:356] "Generic (PLEG): container finished" podID="0357b315-a3ca-4c24-b0e5-51ee48d89983" containerID="4d76cd160f76415e079135eda1f1309d69825c2099a7df0deb78d976f46560fd" exitCode=0
Jul 17 05:39:53.079333 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:53.079185    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6" event={"ID":"0357b315-a3ca-4c24-b0e5-51ee48d89983","Type":"ContainerDied","Data":"4d76cd160f76415e079135eda1f1309d69825c2099a7df0deb78d976f46560fd"}
Jul 17 05:39:53.079622 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:53.079604    2562 scope.go:117] "RemoveContainer" containerID="4d76cd160f76415e079135eda1f1309d69825c2099a7df0deb78d976f46560fd"
Jul 17 05:39:53.080888 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:53.080848    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:39:53.081000 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:53.080947    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:39:53.081047 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:53.081029    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:39:54.089737 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:54.089683    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6" event={"ID":"0357b315-a3ca-4c24-b0e5-51ee48d89983","Type":"ContainerStarted","Data":"27b276c29da6f8840d94182458f6c48e93963c1187d3429537c7e563a01524c2"}
Jul 17 05:39:54.537227 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:54.537138    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6"
Jul 17 05:39:54.537421 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:54.537400    2562 patch_prober.go:28] interesting pod/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Jul 17 05:39:54.537470 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:54.537446    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6" podUID="0357b315-a3ca-4c24-b0e5-51ee48d89983" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Jul 17 05:39:55.538017 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:55.537955    2562 patch_prober.go:28] interesting pod/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Jul 17 05:39:55.538505 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:55.538043    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6" podUID="0357b315-a3ca-4c24-b0e5-51ee48d89983" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Jul 17 05:39:56.537766 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:56.537726    2562 patch_prober.go:28] interesting pod/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Jul 17 05:39:56.537947 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:56.537792    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6" podUID="0357b315-a3ca-4c24-b0e5-51ee48d89983" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Jul 17 05:39:57.537578 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:57.537535    2562 patch_prober.go:28] interesting pod/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Jul 17 05:39:57.537993 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:57.537593    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6" podUID="0357b315-a3ca-4c24-b0e5-51ee48d89983" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Jul 17 05:39:58.540384 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:58.540353    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6"
Jul 17 05:39:58.540796 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:58.540690    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6"
Jul 17 05:39:58.541511 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:58.541491    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6"
Jul 17 05:39:59.120982 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:59.120929    2562 generic.go:356] "Generic (PLEG): container finished" podID="99aa5e8d-f861-41d8-a8db-e764ae921284" containerID="ebc95b438fd814482b1975dc9ffdb71570d5dda8ee21bc833cedaf8c646ba3f0" exitCode=0
Jul 17 05:39:59.121156 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:59.120995    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx" event={"ID":"99aa5e8d-f861-41d8-a8db-e764ae921284","Type":"ContainerDied","Data":"ebc95b438fd814482b1975dc9ffdb71570d5dda8ee21bc833cedaf8c646ba3f0"}
Jul 17 05:39:59.121462 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:59.121444    2562 scope.go:117] "RemoveContainer" containerID="ebc95b438fd814482b1975dc9ffdb71570d5dda8ee21bc833cedaf8c646ba3f0"
Jul 17 05:39:59.122451 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:59.122421    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:39:59.122509 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:59.122492    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:39:59.122560 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:39:59.122524    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 17 05:40:00.128123 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:40:00.128087    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx" event={"ID":"99aa5e8d-f861-41d8-a8db-e764ae921284","Type":"ContainerStarted","Data":"45bbd84517de52c01d9bbf8a431e080ecae77d21dce15d577a056fc6cfe58863"}
Jul 17 05:40:00.414516 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:40:00.414429    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx"
Jul 17 05:40:00.415661 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:40:00.415627    2562 patch_prober.go:28] interesting pod/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Jul 17 05:40:00.415789 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:40:00.415693    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx" podUID="99aa5e8d-f861-41d8-a8db-e764ae921284" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Jul 17 05:40:01.027587 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:40:01.027552    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx"
Jul 17 05:40:01.413867 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:40:01.413836    2562 patch_prober.go:28] interesting pod/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Jul 17 05:40:01.414290 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:40:01.413891    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx" podUID="99aa5e8d-f861-41d8-a8db-e764ae921284" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Jul 17 05:40:02.414867 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:40:02.414824    2562 patch_prober.go:28] interesting pod/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Jul 17 05:40:02.415321 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:40:02.414883    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx" podUID="99aa5e8d-f861-41d8-a8db-e764ae921284" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Jul 17 05:40:03.414814 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:40:03.414777    2562 patch_prober.go:28] interesting pod/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Jul 17 05:40:03.415025 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:40:03.414834    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx" podUID="99aa5e8d-f861-41d8-a8db-e764ae921284" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Jul 17 05:40:04.418432 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:40:04.418400    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx"
Jul 17 05:40:04.419390 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:40:04.419370    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx"
Jul 17 05:40:13.025002 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:40:13.024953    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tx9gj_c1baeee2-350c-43c5-9a04-fb34378edb5c/ovn-acl-logging/0.log"
Jul 17 05:40:13.035187 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:40:13.035163    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tx9gj_c1baeee2-350c-43c5-9a04-fb34378edb5c/ovn-acl-logging/0.log"
Jul 17 05:41:07.230894 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:41:07.230810    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ldn8b"]
Jul 17 05:41:07.233290 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:41:07.233272    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ldn8b"
Jul 17 05:41:07.236026 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:41:07.236004    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 17 05:41:07.236026 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:41:07.236017    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-jhxbd\""
Jul 17 05:41:07.236876 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:41:07.236862    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 17 05:41:07.242326 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:41:07.242301    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ldn8b"]
Jul 17 05:41:07.341360 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:41:07.341326    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6m8p\" (UniqueName: \"kubernetes.io/projected/4427c655-9805-4273-a7fc-0270cd9aace6-kube-api-access-t6m8p\") pod \"kuadrant-operator-catalog-ldn8b\" (UID: \"4427c655-9805-4273-a7fc-0270cd9aace6\") " pod="kuadrant-system/kuadrant-operator-catalog-ldn8b"
Jul 17 05:41:07.442443 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:41:07.442405    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6m8p\" (UniqueName: \"kubernetes.io/projected/4427c655-9805-4273-a7fc-0270cd9aace6-kube-api-access-t6m8p\") pod \"kuadrant-operator-catalog-ldn8b\" (UID: \"4427c655-9805-4273-a7fc-0270cd9aace6\") " pod="kuadrant-system/kuadrant-operator-catalog-ldn8b"
Jul 17 05:41:07.450841 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:41:07.450816    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6m8p\" (UniqueName: \"kubernetes.io/projected/4427c655-9805-4273-a7fc-0270cd9aace6-kube-api-access-t6m8p\") pod \"kuadrant-operator-catalog-ldn8b\" (UID: \"4427c655-9805-4273-a7fc-0270cd9aace6\") " pod="kuadrant-system/kuadrant-operator-catalog-ldn8b"
Jul 17 05:41:07.543496 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:41:07.543401    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ldn8b"
Jul 17 05:41:07.675515 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:41:07.675486    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ldn8b"]
Jul 17 05:41:07.677529 ip-10-0-132-19 kubenswrapper[2562]: W0717 05:41:07.677500    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4427c655_9805_4273_a7fc_0270cd9aace6.slice/crio-eda1953154f7e7abf9c0fc5927a66244be10ecadee0ba50acc0dc50ab09f26d6 WatchSource:0}: Error finding container eda1953154f7e7abf9c0fc5927a66244be10ecadee0ba50acc0dc50ab09f26d6: Status 404 returned error can't find the container with id eda1953154f7e7abf9c0fc5927a66244be10ecadee0ba50acc0dc50ab09f26d6
Jul 17 05:41:08.496771 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:41:08.496733    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ldn8b" event={"ID":"4427c655-9805-4273-a7fc-0270cd9aace6","Type":"ContainerStarted","Data":"eda1953154f7e7abf9c0fc5927a66244be10ecadee0ba50acc0dc50ab09f26d6"}
Jul 17 05:41:10.510144 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:41:10.510107    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ldn8b" event={"ID":"4427c655-9805-4273-a7fc-0270cd9aace6","Type":"ContainerStarted","Data":"475f924cd9ab5b0d575e9aa2d366c772db23e3c72054a77bdae5a23c54d012f4"}
Jul 17 05:41:10.525042 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:41:10.524990    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-ldn8b" podStartSLOduration=1.644024264 podStartE2EDuration="3.524957058s" totalImagesPullingTime="1.880932794s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:41:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:41:10.524029362 +0000 UTC m=+3058.207623046" watchObservedRunningTime="2026-07-17 05:41:10.524957058 +0000 UTC m=+3058.208550714"
Jul 17 05:41:17.543697 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:41:17.543655    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-ldn8b"
Jul 17 05:41:17.544348 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:41:17.543730    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-ldn8b"
Jul 17 05:41:17.567162 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:41:17.567132    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-ldn8b"
Jul 17 05:41:18.573286 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:41:18.573256    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-ldn8b"
Jul 17 05:41:18.605907 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:41:18.605871    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ldn8b"]
Jul 17 05:41:20.568255 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:41:20.568207    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-ldn8b" podUID="4427c655-9805-4273-a7fc-0270cd9aace6" containerName="registry-server" containerID="cri-o://475f924cd9ab5b0d575e9aa2d366c772db23e3c72054a77bdae5a23c54d012f4" gracePeriod=2
Jul 17 05:41:20.820275 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:41:20.820254    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ldn8b"
Jul 17 05:41:20.876905 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:41:20.876882    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t6m8p\" (UniqueName: \"kubernetes.io/projected/4427c655-9805-4273-a7fc-0270cd9aace6-kube-api-access-t6m8p\") pod \"4427c655-9805-4273-a7fc-0270cd9aace6\" (UID: \"4427c655-9805-4273-a7fc-0270cd9aace6\") "
Jul 17 05:41:20.879082 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:41:20.879052    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4427c655-9805-4273-a7fc-0270cd9aace6-kube-api-access-t6m8p" (OuterVolumeSpecName: "kube-api-access-t6m8p") pod "4427c655-9805-4273-a7fc-0270cd9aace6" (UID: "4427c655-9805-4273-a7fc-0270cd9aace6"). InnerVolumeSpecName "kube-api-access-t6m8p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:41:20.978465 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:41:20.978435    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6m8p\" (UniqueName: \"kubernetes.io/projected/4427c655-9805-4273-a7fc-0270cd9aace6-kube-api-access-t6m8p\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:41:21.574159 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:41:21.574121    2562 generic.go:356] "Generic (PLEG): container finished" podID="4427c655-9805-4273-a7fc-0270cd9aace6" containerID="475f924cd9ab5b0d575e9aa2d366c772db23e3c72054a77bdae5a23c54d012f4" exitCode=0
Jul 17 05:41:21.574591 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:41:21.574164    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ldn8b"
Jul 17 05:41:21.574591 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:41:21.574195    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ldn8b" event={"ID":"4427c655-9805-4273-a7fc-0270cd9aace6","Type":"ContainerDied","Data":"475f924cd9ab5b0d575e9aa2d366c772db23e3c72054a77bdae5a23c54d012f4"}
Jul 17 05:41:21.574591 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:41:21.574244    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ldn8b" event={"ID":"4427c655-9805-4273-a7fc-0270cd9aace6","Type":"ContainerDied","Data":"eda1953154f7e7abf9c0fc5927a66244be10ecadee0ba50acc0dc50ab09f26d6"}
Jul 17 05:41:21.574591 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:41:21.574264    2562 scope.go:117] "RemoveContainer" containerID="475f924cd9ab5b0d575e9aa2d366c772db23e3c72054a77bdae5a23c54d012f4"
Jul 17 05:41:21.583497 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:41:21.583480    2562 scope.go:117] "RemoveContainer" containerID="475f924cd9ab5b0d575e9aa2d366c772db23e3c72054a77bdae5a23c54d012f4"
Jul 17 05:41:21.583763 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:41:21.583742    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"475f924cd9ab5b0d575e9aa2d366c772db23e3c72054a77bdae5a23c54d012f4\": container with ID starting with 475f924cd9ab5b0d575e9aa2d366c772db23e3c72054a77bdae5a23c54d012f4 not found: ID does not exist" containerID="475f924cd9ab5b0d575e9aa2d366c772db23e3c72054a77bdae5a23c54d012f4"
Jul 17 05:41:21.583816 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:41:21.583772    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"475f924cd9ab5b0d575e9aa2d366c772db23e3c72054a77bdae5a23c54d012f4"} err="failed to get container status \"475f924cd9ab5b0d575e9aa2d366c772db23e3c72054a77bdae5a23c54d012f4\": rpc error: code = NotFound desc = could not find container \"475f924cd9ab5b0d575e9aa2d366c772db23e3c72054a77bdae5a23c54d012f4\": container with ID starting with 475f924cd9ab5b0d575e9aa2d366c772db23e3c72054a77bdae5a23c54d012f4 not found: ID does not exist"
Jul 17 05:41:21.589463 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:41:21.589439    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ldn8b"]
Jul 17 05:41:21.591227 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:41:21.591208    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ldn8b"]
Jul 17 05:41:22.885862 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:41:22.885827    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4427c655-9805-4273-a7fc-0270cd9aace6" path="/var/lib/kubelet/pods/4427c655-9805-4273-a7fc-0270cd9aace6/volumes"
Jul 17 05:44:27.705592 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:44:27.705509    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-t74hf"]
Jul 17 05:44:27.706065 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:44:27.706006    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4427c655-9805-4273-a7fc-0270cd9aace6" containerName="registry-server"
Jul 17 05:44:27.706065 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:44:27.706020    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4427c655-9805-4273-a7fc-0270cd9aace6" containerName="registry-server"
Jul 17 05:44:27.706141 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:44:27.706110    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4427c655-9805-4273-a7fc-0270cd9aace6" containerName="registry-server"
Jul 17 05:44:27.709278 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:44:27.709262    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-t74hf"
Jul 17 05:44:27.711657 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:44:27.711628    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qh8zj\""
Jul 17 05:44:27.715578 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:44:27.715556    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-t74hf"]
Jul 17 05:44:27.797748 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:44:27.797711    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ad5dec7-fa1a-4ced-9746-c9436d959af5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9eb8486e-t74hf\" (UID: \"0ad5dec7-fa1a-4ced-9746-c9436d959af5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-t74hf"
Jul 17 05:44:27.898663 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:44:27.898621    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ad5dec7-fa1a-4ced-9746-c9436d959af5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9eb8486e-t74hf\" (UID: \"0ad5dec7-fa1a-4ced-9746-c9436d959af5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-t74hf"
Jul 17 05:44:27.899279 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:44:27.899259    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ad5dec7-fa1a-4ced-9746-c9436d959af5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9eb8486e-t74hf\" (UID: \"0ad5dec7-fa1a-4ced-9746-c9436d959af5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-t74hf"
Jul 17 05:44:28.021576 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:44:28.021494    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-t74hf"
Jul 17 05:44:28.156119 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:44:28.156090    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-t74hf"]
Jul 17 05:44:28.158213 ip-10-0-132-19 kubenswrapper[2562]: W0717 05:44:28.158174    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ad5dec7_fa1a_4ced_9746_c9436d959af5.slice/crio-28cac4fdc6c2e2bf924cbad0eaceedaafe7933c76f3fb8b5ce7149bf9f4db299 WatchSource:0}: Error finding container 28cac4fdc6c2e2bf924cbad0eaceedaafe7933c76f3fb8b5ce7149bf9f4db299: Status 404 returned error can't find the container with id 28cac4fdc6c2e2bf924cbad0eaceedaafe7933c76f3fb8b5ce7149bf9f4db299
Jul 17 05:44:28.524369 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:44:28.524334    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-t74hf" event={"ID":"0ad5dec7-fa1a-4ced-9746-c9436d959af5","Type":"ContainerStarted","Data":"56a7dd33ecb6d1a034bed41d48e1a5f6cb2729b6abbc390459f0e9b5d69257aa"}
Jul 17 05:44:28.524569 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:44:28.524390    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-t74hf" event={"ID":"0ad5dec7-fa1a-4ced-9746-c9436d959af5","Type":"ContainerStarted","Data":"28cac4fdc6c2e2bf924cbad0eaceedaafe7933c76f3fb8b5ce7149bf9f4db299"}
Jul 17 05:44:28.540501 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:44:28.540407    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-t74hf" podStartSLOduration=1.5403912229999999 podStartE2EDuration="1.540391223s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:44:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:44:28.538236021 +0000 UTC m=+3256.221829677" watchObservedRunningTime="2026-07-17 05:44:28.540391223 +0000 UTC m=+3256.223984880"
Jul 17 05:44:29.532073 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:44:29.532037    2562 generic.go:356] "Generic (PLEG): container finished" podID="0ad5dec7-fa1a-4ced-9746-c9436d959af5" containerID="56a7dd33ecb6d1a034bed41d48e1a5f6cb2729b6abbc390459f0e9b5d69257aa" exitCode=6
Jul 17 05:44:29.532497 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:44:29.532108    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-t74hf" event={"ID":"0ad5dec7-fa1a-4ced-9746-c9436d959af5","Type":"ContainerDied","Data":"56a7dd33ecb6d1a034bed41d48e1a5f6cb2729b6abbc390459f0e9b5d69257aa"}
Jul 17 05:44:29.532497 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:44:29.532264    2562 scope.go:117] "RemoveContainer" containerID="56a7dd33ecb6d1a034bed41d48e1a5f6cb2729b6abbc390459f0e9b5d69257aa"
Jul 17 05:44:30.538883 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:44:30.538785    2562 generic.go:356] "Generic (PLEG): container finished" podID="0ad5dec7-fa1a-4ced-9746-c9436d959af5" containerID="c40ad71d679f3c3d603973257136695d3eb949ca69c430463f865c3cb680d84e" exitCode=6
Jul 17 05:44:30.538883 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:44:30.538861    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-t74hf" event={"ID":"0ad5dec7-fa1a-4ced-9746-c9436d959af5","Type":"ContainerDied","Data":"c40ad71d679f3c3d603973257136695d3eb949ca69c430463f865c3cb680d84e"}
Jul 17 05:44:30.539388 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:44:30.538911    2562 scope.go:117] "RemoveContainer" containerID="56a7dd33ecb6d1a034bed41d48e1a5f6cb2729b6abbc390459f0e9b5d69257aa"
Jul 17 05:44:30.539388 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:44:30.539141    2562 scope.go:117] "RemoveContainer" containerID="c40ad71d679f3c3d603973257136695d3eb949ca69c430463f865c3cb680d84e"
Jul 17 05:44:30.539462 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:44:30.539391    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-9eb8486e-t74hf_odh-ai-gateway-infra(0ad5dec7-fa1a-4ced-9746-c9436d959af5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-t74hf" podUID="0ad5dec7-fa1a-4ced-9746-c9436d959af5"
Jul 17 05:44:31.545261 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:44:31.545226    2562 scope.go:117] "RemoveContainer" containerID="c40ad71d679f3c3d603973257136695d3eb949ca69c430463f865c3cb680d84e"
Jul 17 05:44:31.545656 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:44:31.545426    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-9eb8486e-t74hf_odh-ai-gateway-infra(0ad5dec7-fa1a-4ced-9746-c9436d959af5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-t74hf" podUID="0ad5dec7-fa1a-4ced-9746-c9436d959af5"
Jul 17 05:44:42.880799 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:44:42.880769    2562 scope.go:117] "RemoveContainer" containerID="c40ad71d679f3c3d603973257136695d3eb949ca69c430463f865c3cb680d84e"
Jul 17 05:44:43.613939 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:44:43.613896    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-t74hf" event={"ID":"0ad5dec7-fa1a-4ced-9746-c9436d959af5","Type":"ContainerStarted","Data":"c36fb778d98b2463229927c48cd84682105fdd22710ff90eabe44cd16b2de31f"}
Jul 17 05:44:44.620796 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:44:44.620762    2562 generic.go:356] "Generic (PLEG): container finished" podID="0ad5dec7-fa1a-4ced-9746-c9436d959af5" containerID="c36fb778d98b2463229927c48cd84682105fdd22710ff90eabe44cd16b2de31f" exitCode=6
Jul 17 05:44:44.621234 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:44:44.620841    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-t74hf" event={"ID":"0ad5dec7-fa1a-4ced-9746-c9436d959af5","Type":"ContainerDied","Data":"c36fb778d98b2463229927c48cd84682105fdd22710ff90eabe44cd16b2de31f"}
Jul 17 05:44:44.621234 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:44:44.620907    2562 scope.go:117] "RemoveContainer" containerID="c40ad71d679f3c3d603973257136695d3eb949ca69c430463f865c3cb680d84e"
Jul 17 05:44:44.621341 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:44:44.621232    2562 scope.go:117] "RemoveContainer" containerID="c36fb778d98b2463229927c48cd84682105fdd22710ff90eabe44cd16b2de31f"
Jul 17 05:44:44.621549 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:44:44.621524    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-9eb8486e-t74hf_odh-ai-gateway-infra(0ad5dec7-fa1a-4ced-9746-c9436d959af5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-t74hf" podUID="0ad5dec7-fa1a-4ced-9746-c9436d959af5"
Jul 17 05:44:44.644954 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:44:44.644920    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-t74hf"]
Jul 17 05:44:45.762008 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:44:45.761947    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-t74hf"
Jul 17 05:44:45.883577 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:44:45.883486    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ad5dec7-fa1a-4ced-9746-c9436d959af5-maas-api-service-ca\") pod \"0ad5dec7-fa1a-4ced-9746-c9436d959af5\" (UID: \"0ad5dec7-fa1a-4ced-9746-c9436d959af5\") "
Jul 17 05:44:45.883869 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:44:45.883846    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ad5dec7-fa1a-4ced-9746-c9436d959af5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0ad5dec7-fa1a-4ced-9746-c9436d959af5" (UID: "0ad5dec7-fa1a-4ced-9746-c9436d959af5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:44:45.985423 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:44:45.985379    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ad5dec7-fa1a-4ced-9746-c9436d959af5-maas-api-service-ca\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:44:46.633231 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:44:46.633199    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-t74hf"
Jul 17 05:44:46.633435 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:44:46.633233    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-t74hf" event={"ID":"0ad5dec7-fa1a-4ced-9746-c9436d959af5","Type":"ContainerDied","Data":"28cac4fdc6c2e2bf924cbad0eaceedaafe7933c76f3fb8b5ce7149bf9f4db299"}
Jul 17 05:44:46.633435 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:44:46.633295    2562 scope.go:117] "RemoveContainer" containerID="c36fb778d98b2463229927c48cd84682105fdd22710ff90eabe44cd16b2de31f"
Jul 17 05:44:46.655476 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:44:46.655444    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-t74hf"]
Jul 17 05:44:46.663179 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:44:46.663155    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-t74hf"]
Jul 17 05:44:46.885099 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:44:46.885005    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ad5dec7-fa1a-4ced-9746-c9436d959af5" path="/var/lib/kubelet/pods/0ad5dec7-fa1a-4ced-9746-c9436d959af5/volumes"
Jul 17 05:45:00.131885 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:00.131761    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-823cda-29737785-8bnxb"]
Jul 17 05:45:00.134401 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:00.132387    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ad5dec7-fa1a-4ced-9746-c9436d959af5" containerName="revoke-keys"
Jul 17 05:45:00.134401 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:00.132403    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ad5dec7-fa1a-4ced-9746-c9436d959af5" containerName="revoke-keys"
Jul 17 05:45:00.134401 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:00.132412    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ad5dec7-fa1a-4ced-9746-c9436d959af5" containerName="revoke-keys"
Jul 17 05:45:00.134401 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:00.132417    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ad5dec7-fa1a-4ced-9746-c9436d959af5" containerName="revoke-keys"
Jul 17 05:45:00.134401 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:00.132436    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ad5dec7-fa1a-4ced-9746-c9436d959af5" containerName="revoke-keys"
Jul 17 05:45:00.134401 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:00.132441    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ad5dec7-fa1a-4ced-9746-c9436d959af5" containerName="revoke-keys"
Jul 17 05:45:00.134401 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:00.132505    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ad5dec7-fa1a-4ced-9746-c9436d959af5" containerName="revoke-keys"
Jul 17 05:45:00.134401 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:00.132517    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ad5dec7-fa1a-4ced-9746-c9436d959af5" containerName="revoke-keys"
Jul 17 05:45:00.135465 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:00.135446    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737785-kdjmj"]
Jul 17 05:45:00.135574 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:00.135555    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-823cda-29737785-8bnxb"
Jul 17 05:45:00.135938 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:00.135912    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ad5dec7-fa1a-4ced-9746-c9436d959af5" containerName="revoke-keys"
Jul 17 05:45:00.138200 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:00.138181    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qh8zj\""
Jul 17 05:45:00.139035 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:00.138950    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-560f5b-29737785-zd6zc"]
Jul 17 05:45:00.139149 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:00.139086    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737785-kdjmj"
Jul 17 05:45:00.146037 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:00.142839    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-560f5b-29737785-zd6zc"
Jul 17 05:45:00.146037 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:00.142857    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737785-kdjmj"]
Jul 17 05:45:00.146824 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:00.146797    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-823cda-29737785-8bnxb"]
Jul 17 05:45:00.150061 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:00.150040    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-560f5b-29737785-zd6zc"]
Jul 17 05:45:00.152195 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:00.152177    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-823cda-29737785-8bnxb"
Jul 17 05:45:00.162097 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:00.162076    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737785-kdjmj"
Jul 17 05:45:00.197219 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:00.197178    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-560f5b-29737785-zd6zc"
Jul 17 05:45:00.348331 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:00.347857    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737785-kdjmj"]
Jul 17 05:45:00.350447 ip-10-0-132-19 kubenswrapper[2562]: W0717 05:45:00.350421    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8817732_a28b_466e_9c7d_c68a72050030.slice/crio-d22737dbfbe88413198a3d2e1944c1ba513b4c538325e5863e469168bcbea7b9 WatchSource:0}: Error finding container d22737dbfbe88413198a3d2e1944c1ba513b4c538325e5863e469168bcbea7b9: Status 404 returned error can't find the container with id d22737dbfbe88413198a3d2e1944c1ba513b4c538325e5863e469168bcbea7b9
Jul 17 05:45:00.376742 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:00.376715    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-560f5b-29737785-zd6zc"]
Jul 17 05:45:00.387143 ip-10-0-132-19 kubenswrapper[2562]: W0717 05:45:00.387114    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc23c32f2_830f_4f5a_89aa_ef53e8b0eeb8.slice/crio-42ced6ddf63028b7fdf139ff4562722cfc61924ee814f8aa683866fcd5f92db6 WatchSource:0}: Error finding container 42ced6ddf63028b7fdf139ff4562722cfc61924ee814f8aa683866fcd5f92db6: Status 404 returned error can't find the container with id 42ced6ddf63028b7fdf139ff4562722cfc61924ee814f8aa683866fcd5f92db6
Jul 17 05:45:00.528161 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:00.528097    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-823cda-29737785-8bnxb"]
Jul 17 05:45:00.530345 ip-10-0-132-19 kubenswrapper[2562]: W0717 05:45:00.530323    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod356830d7_dbe1_4b96_b025_232e35a1a70b.slice/crio-bd8fbb4978ee2947b33b9e3969c7e5c9b1c3f8a6ef1cd983022a767507655252 WatchSource:0}: Error finding container bd8fbb4978ee2947b33b9e3969c7e5c9b1c3f8a6ef1cd983022a767507655252: Status 404 returned error can't find the container with id bd8fbb4978ee2947b33b9e3969c7e5c9b1c3f8a6ef1cd983022a767507655252
Jul 17 05:45:00.712707 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:00.712670    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-560f5b-29737785-zd6zc" event={"ID":"c23c32f2-830f-4f5a-89aa-ef53e8b0eeb8","Type":"ContainerStarted","Data":"6e2ca8dbf5601cf2dada807fd2ca61f4bc77cee0f094d556860929d70a9e69cc"}
Jul 17 05:45:00.712896 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:00.712769    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-560f5b-29737785-zd6zc" event={"ID":"c23c32f2-830f-4f5a-89aa-ef53e8b0eeb8","Type":"ContainerStarted","Data":"42ced6ddf63028b7fdf139ff4562722cfc61924ee814f8aa683866fcd5f92db6"}
Jul 17 05:45:00.714176 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:00.714148    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737785-kdjmj" event={"ID":"a8817732-a28b-466e-9c7d-c68a72050030","Type":"ContainerStarted","Data":"0fc6410eb53357566ccad56d5e281ebff8d51040b51a69a050923875965ac39b"}
Jul 17 05:45:00.714309 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:00.714199    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737785-kdjmj" event={"ID":"a8817732-a28b-466e-9c7d-c68a72050030","Type":"ContainerStarted","Data":"d22737dbfbe88413198a3d2e1944c1ba513b4c538325e5863e469168bcbea7b9"}
Jul 17 05:45:00.715382 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:00.715352    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-823cda-29737785-8bnxb" event={"ID":"356830d7-dbe1-4b96-b025-232e35a1a70b","Type":"ContainerStarted","Data":"bd8fbb4978ee2947b33b9e3969c7e5c9b1c3f8a6ef1cd983022a767507655252"}
Jul 17 05:45:00.728184 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:00.728117    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-560f5b-29737785-zd6zc" podStartSLOduration=0.728102723 podStartE2EDuration="728.102723ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:45:00.725515538 +0000 UTC m=+3288.409109195" watchObservedRunningTime="2026-07-17 05:45:00.728102723 +0000 UTC m=+3288.411696381"
Jul 17 05:45:00.739611 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:00.739563    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737785-kdjmj" podStartSLOduration=0.739549197 podStartE2EDuration="739.549197ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:45:00.73840459 +0000 UTC m=+3288.421998260" watchObservedRunningTime="2026-07-17 05:45:00.739549197 +0000 UTC m=+3288.423142832"
Jul 17 05:45:01.723744 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:01.723706    2562 generic.go:356] "Generic (PLEG): container finished" podID="c23c32f2-830f-4f5a-89aa-ef53e8b0eeb8" containerID="6e2ca8dbf5601cf2dada807fd2ca61f4bc77cee0f094d556860929d70a9e69cc" exitCode=0
Jul 17 05:45:01.724226 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:01.723790    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-560f5b-29737785-zd6zc" event={"ID":"c23c32f2-830f-4f5a-89aa-ef53e8b0eeb8","Type":"ContainerDied","Data":"6e2ca8dbf5601cf2dada807fd2ca61f4bc77cee0f094d556860929d70a9e69cc"}
Jul 17 05:45:01.725362 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:01.725326    2562 generic.go:356] "Generic (PLEG): container finished" podID="a8817732-a28b-466e-9c7d-c68a72050030" containerID="0fc6410eb53357566ccad56d5e281ebff8d51040b51a69a050923875965ac39b" exitCode=0
Jul 17 05:45:01.725469 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:01.725403    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737785-kdjmj" event={"ID":"a8817732-a28b-466e-9c7d-c68a72050030","Type":"ContainerDied","Data":"0fc6410eb53357566ccad56d5e281ebff8d51040b51a69a050923875965ac39b"}
Jul 17 05:45:01.726994 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:01.726952    2562 generic.go:356] "Generic (PLEG): container finished" podID="356830d7-dbe1-4b96-b025-232e35a1a70b" containerID="a67792675db80c7cbc4c6ee93fa3325da110e79ca5ccac1187e8a46fce914d76" exitCode=0
Jul 17 05:45:01.727075 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:01.726990    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-823cda-29737785-8bnxb" event={"ID":"356830d7-dbe1-4b96-b025-232e35a1a70b","Type":"ContainerDied","Data":"a67792675db80c7cbc4c6ee93fa3325da110e79ca5ccac1187e8a46fce914d76"}
Jul 17 05:45:02.886388 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:02.886310    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-560f5b-29737785-zd6zc"
Jul 17 05:45:02.924089 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:02.924062    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-823cda-29737785-8bnxb"
Jul 17 05:45:02.927525 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:02.927488    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737785-kdjmj"
Jul 17 05:45:03.738592 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:03.738565    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-560f5b-29737785-zd6zc"
Jul 17 05:45:03.738592 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:03.738582    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-560f5b-29737785-zd6zc" event={"ID":"c23c32f2-830f-4f5a-89aa-ef53e8b0eeb8","Type":"ContainerDied","Data":"42ced6ddf63028b7fdf139ff4562722cfc61924ee814f8aa683866fcd5f92db6"}
Jul 17 05:45:03.738848 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:03.738638    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="42ced6ddf63028b7fdf139ff4562722cfc61924ee814f8aa683866fcd5f92db6"
Jul 17 05:45:03.739942 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:03.739920    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737785-kdjmj"
Jul 17 05:45:03.740099 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:03.739922    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737785-kdjmj" event={"ID":"a8817732-a28b-466e-9c7d-c68a72050030","Type":"ContainerDied","Data":"d22737dbfbe88413198a3d2e1944c1ba513b4c538325e5863e469168bcbea7b9"}
Jul 17 05:45:03.740099 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:03.740014    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d22737dbfbe88413198a3d2e1944c1ba513b4c538325e5863e469168bcbea7b9"
Jul 17 05:45:03.741493 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:03.741458    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-823cda-29737785-8bnxb"
Jul 17 05:45:03.741629 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:03.741513    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-823cda-29737785-8bnxb" event={"ID":"356830d7-dbe1-4b96-b025-232e35a1a70b","Type":"ContainerDied","Data":"bd8fbb4978ee2947b33b9e3969c7e5c9b1c3f8a6ef1cd983022a767507655252"}
Jul 17 05:45:03.741629 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:03.741555    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bd8fbb4978ee2947b33b9e3969c7e5c9b1c3f8a6ef1cd983022a767507655252"
Jul 17 05:45:11.512500 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:11.512460    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-f74fg"]
Jul 17 05:45:11.512955 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:11.512940    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="356830d7-dbe1-4b96-b025-232e35a1a70b" containerName="cleanup"
Jul 17 05:45:11.512955 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:11.512956    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="356830d7-dbe1-4b96-b025-232e35a1a70b" containerName="cleanup"
Jul 17 05:45:11.512955 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:11.512984    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8817732-a28b-466e-9c7d-c68a72050030" containerName="cleanup"
Jul 17 05:45:11.513114 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:11.512990    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8817732-a28b-466e-9c7d-c68a72050030" containerName="cleanup"
Jul 17 05:45:11.513114 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:11.513005    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c23c32f2-830f-4f5a-89aa-ef53e8b0eeb8" containerName="cleanup"
Jul 17 05:45:11.513114 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:11.513010    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c23c32f2-830f-4f5a-89aa-ef53e8b0eeb8" containerName="cleanup"
Jul 17 05:45:11.513114 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:11.513072    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c23c32f2-830f-4f5a-89aa-ef53e8b0eeb8" containerName="cleanup"
Jul 17 05:45:11.513114 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:11.513082    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8817732-a28b-466e-9c7d-c68a72050030" containerName="cleanup"
Jul 17 05:45:11.513114 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:11.513092    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="356830d7-dbe1-4b96-b025-232e35a1a70b" containerName="cleanup"
Jul 17 05:45:11.516297 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:11.516281    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-f74fg"
Jul 17 05:45:11.519026 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:11.519003    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qh8zj\""
Jul 17 05:45:11.525185 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:11.525132    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-f74fg"]
Jul 17 05:45:11.634472 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:11.634437    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4132650c-801d-42c3-9c48-93ce44c8351e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4e221674-f74fg\" (UID: \"4132650c-801d-42c3-9c48-93ce44c8351e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-f74fg"
Jul 17 05:45:11.735546 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:11.735513    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4132650c-801d-42c3-9c48-93ce44c8351e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4e221674-f74fg\" (UID: \"4132650c-801d-42c3-9c48-93ce44c8351e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-f74fg"
Jul 17 05:45:11.736152 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:11.736134    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4132650c-801d-42c3-9c48-93ce44c8351e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4e221674-f74fg\" (UID: \"4132650c-801d-42c3-9c48-93ce44c8351e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-f74fg"
Jul 17 05:45:11.828143 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:11.828112    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-f74fg"
Jul 17 05:45:11.967532 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:11.967507    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-f74fg"]
Jul 17 05:45:11.970471 ip-10-0-132-19 kubenswrapper[2562]: W0717 05:45:11.970441    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4132650c_801d_42c3_9c48_93ce44c8351e.slice/crio-4f9300fcb9dec1647f2f53eaf3d46aba645e7c42b22f93f3d669dcf35ec01e6e WatchSource:0}: Error finding container 4f9300fcb9dec1647f2f53eaf3d46aba645e7c42b22f93f3d669dcf35ec01e6e: Status 404 returned error can't find the container with id 4f9300fcb9dec1647f2f53eaf3d46aba645e7c42b22f93f3d669dcf35ec01e6e
Jul 17 05:45:12.789659 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:12.789619    2562 generic.go:356] "Generic (PLEG): container finished" podID="4132650c-801d-42c3-9c48-93ce44c8351e" containerID="3809fe2059059cc49efa08058ef0f9cd51e09985ba2419168ec0ea0ba1ebb303" exitCode=6
Jul 17 05:45:12.790094 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:12.789713    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-f74fg" event={"ID":"4132650c-801d-42c3-9c48-93ce44c8351e","Type":"ContainerDied","Data":"3809fe2059059cc49efa08058ef0f9cd51e09985ba2419168ec0ea0ba1ebb303"}
Jul 17 05:45:12.790094 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:12.789777    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-f74fg" event={"ID":"4132650c-801d-42c3-9c48-93ce44c8351e","Type":"ContainerStarted","Data":"4f9300fcb9dec1647f2f53eaf3d46aba645e7c42b22f93f3d669dcf35ec01e6e"}
Jul 17 05:45:12.790094 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:12.789903    2562 scope.go:117] "RemoveContainer" containerID="3809fe2059059cc49efa08058ef0f9cd51e09985ba2419168ec0ea0ba1ebb303"
Jul 17 05:45:13.067657 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:13.067633    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tx9gj_c1baeee2-350c-43c5-9a04-fb34378edb5c/ovn-acl-logging/0.log"
Jul 17 05:45:13.072438 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:13.072410    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tx9gj_c1baeee2-350c-43c5-9a04-fb34378edb5c/ovn-acl-logging/0.log"
Jul 17 05:45:13.796213 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:13.796177    2562 generic.go:356] "Generic (PLEG): container finished" podID="4132650c-801d-42c3-9c48-93ce44c8351e" containerID="27a813b30ba00785b2f6cdb5e6950e6e58f07a32df912370c194beac12050465" exitCode=6
Jul 17 05:45:13.796652 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:13.796258    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-f74fg" event={"ID":"4132650c-801d-42c3-9c48-93ce44c8351e","Type":"ContainerDied","Data":"27a813b30ba00785b2f6cdb5e6950e6e58f07a32df912370c194beac12050465"}
Jul 17 05:45:13.796652 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:13.796309    2562 scope.go:117] "RemoveContainer" containerID="3809fe2059059cc49efa08058ef0f9cd51e09985ba2419168ec0ea0ba1ebb303"
Jul 17 05:45:13.796652 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:13.796500    2562 scope.go:117] "RemoveContainer" containerID="27a813b30ba00785b2f6cdb5e6950e6e58f07a32df912370c194beac12050465"
Jul 17 05:45:13.796789 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:45:13.796759    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-4e221674-f74fg_odh-ai-gateway-infra(4132650c-801d-42c3-9c48-93ce44c8351e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-f74fg" podUID="4132650c-801d-42c3-9c48-93ce44c8351e"
Jul 17 05:45:14.803126 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:14.803102    2562 scope.go:117] "RemoveContainer" containerID="27a813b30ba00785b2f6cdb5e6950e6e58f07a32df912370c194beac12050465"
Jul 17 05:45:14.803587 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:45:14.803372    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-4e221674-f74fg_odh-ai-gateway-infra(4132650c-801d-42c3-9c48-93ce44c8351e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-f74fg" podUID="4132650c-801d-42c3-9c48-93ce44c8351e"
Jul 17 05:45:28.878209 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:28.878173    2562 scope.go:117] "RemoveContainer" containerID="27a813b30ba00785b2f6cdb5e6950e6e58f07a32df912370c194beac12050465"
Jul 17 05:45:29.259169 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:29.259121    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-560f5b-29737785-zd6zc"]
Jul 17 05:45:29.261289 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:29.261259    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-560f5b-29737785-zd6zc"]
Jul 17 05:45:29.884015 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:29.883957    2562 generic.go:356] "Generic (PLEG): container finished" podID="4132650c-801d-42c3-9c48-93ce44c8351e" containerID="5cc1235a72bfc172f664b6371578d59e8731cf04d0e685b5a5761c2abf8c1488" exitCode=6
Jul 17 05:45:29.884015 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:29.883996    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-f74fg" event={"ID":"4132650c-801d-42c3-9c48-93ce44c8351e","Type":"ContainerDied","Data":"5cc1235a72bfc172f664b6371578d59e8731cf04d0e685b5a5761c2abf8c1488"}
Jul 17 05:45:29.884516 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:29.884057    2562 scope.go:117] "RemoveContainer" containerID="27a813b30ba00785b2f6cdb5e6950e6e58f07a32df912370c194beac12050465"
Jul 17 05:45:29.884516 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:29.884366    2562 scope.go:117] "RemoveContainer" containerID="5cc1235a72bfc172f664b6371578d59e8731cf04d0e685b5a5761c2abf8c1488"
Jul 17 05:45:29.884604 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:45:29.884577    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-4e221674-f74fg_odh-ai-gateway-infra(4132650c-801d-42c3-9c48-93ce44c8351e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-f74fg" podUID="4132650c-801d-42c3-9c48-93ce44c8351e"
Jul 17 05:45:30.884890 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:30.884850    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c23c32f2-830f-4f5a-89aa-ef53e8b0eeb8" path="/var/lib/kubelet/pods/c23c32f2-830f-4f5a-89aa-ef53e8b0eeb8/volumes"
Jul 17 05:45:30.911309 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:30.911275    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-f74fg"]
Jul 17 05:45:31.052526 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:31.052498    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-f74fg"
Jul 17 05:45:31.224788 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:31.224703    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4132650c-801d-42c3-9c48-93ce44c8351e-maas-api-service-ca\") pod \"4132650c-801d-42c3-9c48-93ce44c8351e\" (UID: \"4132650c-801d-42c3-9c48-93ce44c8351e\") "
Jul 17 05:45:31.225077 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:31.225054    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4132650c-801d-42c3-9c48-93ce44c8351e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4132650c-801d-42c3-9c48-93ce44c8351e" (UID: "4132650c-801d-42c3-9c48-93ce44c8351e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:45:31.326331 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:31.326295    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4132650c-801d-42c3-9c48-93ce44c8351e-maas-api-service-ca\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:45:31.897563 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:31.897529    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-f74fg"
Jul 17 05:45:31.898059 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:31.897585    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-f74fg" event={"ID":"4132650c-801d-42c3-9c48-93ce44c8351e","Type":"ContainerDied","Data":"4f9300fcb9dec1647f2f53eaf3d46aba645e7c42b22f93f3d669dcf35ec01e6e"}
Jul 17 05:45:31.898059 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:31.897638    2562 scope.go:117] "RemoveContainer" containerID="5cc1235a72bfc172f664b6371578d59e8731cf04d0e685b5a5761c2abf8c1488"
Jul 17 05:45:31.922264 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:31.922229    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-f74fg"]
Jul 17 05:45:31.926170 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:31.926146    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4e221674-f74fg"]
Jul 17 05:45:32.884309 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:32.884257    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4132650c-801d-42c3-9c48-93ce44c8351e" path="/var/lib/kubelet/pods/4132650c-801d-42c3-9c48-93ce44c8351e/volumes"
Jul 17 05:45:44.516008 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:44.515951    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6"]
Jul 17 05:45:44.516461 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:44.516220    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6" podUID="0357b315-a3ca-4c24-b0e5-51ee48d89983" containerName="istio-proxy" containerID="cri-o://27b276c29da6f8840d94182458f6c48e93963c1187d3429537c7e563a01524c2" gracePeriod=30
Jul 17 05:45:49.766144 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:49.766117    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6"
Jul 17 05:45:49.808075 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:49.807993    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8qwgw\" (UniqueName: \"kubernetes.io/projected/0357b315-a3ca-4c24-b0e5-51ee48d89983-kube-api-access-8qwgw\") pod \"0357b315-a3ca-4c24-b0e5-51ee48d89983\" (UID: \"0357b315-a3ca-4c24-b0e5-51ee48d89983\") "
Jul 17 05:45:49.808219 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:49.808087    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0357b315-a3ca-4c24-b0e5-51ee48d89983-workload-certs\") pod \"0357b315-a3ca-4c24-b0e5-51ee48d89983\" (UID: \"0357b315-a3ca-4c24-b0e5-51ee48d89983\") "
Jul 17 05:45:49.808219 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:49.808165    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0357b315-a3ca-4c24-b0e5-51ee48d89983-workload-socket\") pod \"0357b315-a3ca-4c24-b0e5-51ee48d89983\" (UID: \"0357b315-a3ca-4c24-b0e5-51ee48d89983\") "
Jul 17 05:45:49.808219 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:49.808189    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0357b315-a3ca-4c24-b0e5-51ee48d89983-istio-token\") pod \"0357b315-a3ca-4c24-b0e5-51ee48d89983\" (UID: \"0357b315-a3ca-4c24-b0e5-51ee48d89983\") "
Jul 17 05:45:49.808396 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:49.808244    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0357b315-a3ca-4c24-b0e5-51ee48d89983-istio-podinfo\") pod \"0357b315-a3ca-4c24-b0e5-51ee48d89983\" (UID: \"0357b315-a3ca-4c24-b0e5-51ee48d89983\") "
Jul 17 05:45:49.808396 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:49.808280    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0357b315-a3ca-4c24-b0e5-51ee48d89983-credential-socket\") pod \"0357b315-a3ca-4c24-b0e5-51ee48d89983\" (UID: \"0357b315-a3ca-4c24-b0e5-51ee48d89983\") "
Jul 17 05:45:49.808396 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:49.808323    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0357b315-a3ca-4c24-b0e5-51ee48d89983-istio-data\") pod \"0357b315-a3ca-4c24-b0e5-51ee48d89983\" (UID: \"0357b315-a3ca-4c24-b0e5-51ee48d89983\") "
Jul 17 05:45:49.808396 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:49.808360    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0357b315-a3ca-4c24-b0e5-51ee48d89983-istio-envoy\") pod \"0357b315-a3ca-4c24-b0e5-51ee48d89983\" (UID: \"0357b315-a3ca-4c24-b0e5-51ee48d89983\") "
Jul 17 05:45:49.808607 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:49.808402    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0357b315-a3ca-4c24-b0e5-51ee48d89983-istiod-ca-cert\") pod \"0357b315-a3ca-4c24-b0e5-51ee48d89983\" (UID: \"0357b315-a3ca-4c24-b0e5-51ee48d89983\") "
Jul 17 05:45:49.808607 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:49.808377    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0357b315-a3ca-4c24-b0e5-51ee48d89983-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0357b315-a3ca-4c24-b0e5-51ee48d89983" (UID: "0357b315-a3ca-4c24-b0e5-51ee48d89983"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:45:49.808607 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:49.808463    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0357b315-a3ca-4c24-b0e5-51ee48d89983-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0357b315-a3ca-4c24-b0e5-51ee48d89983" (UID: "0357b315-a3ca-4c24-b0e5-51ee48d89983"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:45:49.808757 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:49.808620    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0357b315-a3ca-4c24-b0e5-51ee48d89983-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0357b315-a3ca-4c24-b0e5-51ee48d89983" (UID: "0357b315-a3ca-4c24-b0e5-51ee48d89983"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:45:49.808757 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:49.808730    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0357b315-a3ca-4c24-b0e5-51ee48d89983-workload-certs\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:45:49.808757 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:49.808747    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0357b315-a3ca-4c24-b0e5-51ee48d89983-workload-socket\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:45:49.809108 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:49.808759    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0357b315-a3ca-4c24-b0e5-51ee48d89983-credential-socket\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:45:49.809108 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:49.808928    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0357b315-a3ca-4c24-b0e5-51ee48d89983-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0357b315-a3ca-4c24-b0e5-51ee48d89983" (UID: "0357b315-a3ca-4c24-b0e5-51ee48d89983"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:45:49.809545 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:49.809514    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0357b315-a3ca-4c24-b0e5-51ee48d89983-istio-data" (OuterVolumeSpecName: "istio-data") pod "0357b315-a3ca-4c24-b0e5-51ee48d89983" (UID: "0357b315-a3ca-4c24-b0e5-51ee48d89983"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:45:49.810715 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:49.810687    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0357b315-a3ca-4c24-b0e5-51ee48d89983-kube-api-access-8qwgw" (OuterVolumeSpecName: "kube-api-access-8qwgw") pod "0357b315-a3ca-4c24-b0e5-51ee48d89983" (UID: "0357b315-a3ca-4c24-b0e5-51ee48d89983"). InnerVolumeSpecName "kube-api-access-8qwgw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:45:49.810822 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:49.810740    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0357b315-a3ca-4c24-b0e5-51ee48d89983-istio-token" (OuterVolumeSpecName: "istio-token") pod "0357b315-a3ca-4c24-b0e5-51ee48d89983" (UID: "0357b315-a3ca-4c24-b0e5-51ee48d89983"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:45:49.811093 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:49.811067    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0357b315-a3ca-4c24-b0e5-51ee48d89983-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0357b315-a3ca-4c24-b0e5-51ee48d89983" (UID: "0357b315-a3ca-4c24-b0e5-51ee48d89983"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:45:49.811187 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:49.811087    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0357b315-a3ca-4c24-b0e5-51ee48d89983-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0357b315-a3ca-4c24-b0e5-51ee48d89983" (UID: "0357b315-a3ca-4c24-b0e5-51ee48d89983"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 05:45:49.909599 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:49.909561    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0357b315-a3ca-4c24-b0e5-51ee48d89983-istio-token\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:45:49.909599 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:49.909592    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0357b315-a3ca-4c24-b0e5-51ee48d89983-istio-podinfo\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:45:49.909599 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:49.909600    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0357b315-a3ca-4c24-b0e5-51ee48d89983-istio-data\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:45:49.909599 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:49.909608    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0357b315-a3ca-4c24-b0e5-51ee48d89983-istio-envoy\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:45:49.909863 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:49.909617    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0357b315-a3ca-4c24-b0e5-51ee48d89983-istiod-ca-cert\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:45:49.909863 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:49.909625    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8qwgw\" (UniqueName: \"kubernetes.io/projected/0357b315-a3ca-4c24-b0e5-51ee48d89983-kube-api-access-8qwgw\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:45:49.993457 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:49.993423    2562 generic.go:356] "Generic (PLEG): container finished" podID="0357b315-a3ca-4c24-b0e5-51ee48d89983" containerID="27b276c29da6f8840d94182458f6c48e93963c1187d3429537c7e563a01524c2" exitCode=0
Jul 17 05:45:49.993628 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:49.993468    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6" event={"ID":"0357b315-a3ca-4c24-b0e5-51ee48d89983","Type":"ContainerDied","Data":"27b276c29da6f8840d94182458f6c48e93963c1187d3429537c7e563a01524c2"}
Jul 17 05:45:49.993628 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:49.993471    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6"
Jul 17 05:45:49.993628 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:49.993511    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6" event={"ID":"0357b315-a3ca-4c24-b0e5-51ee48d89983","Type":"ContainerDied","Data":"44b5a9ca2352f7953469bfbd163a0cd45e5a3c93838759e4b69d59b45be79af0"}
Jul 17 05:45:49.993628 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:49.993532    2562 scope.go:117] "RemoveContainer" containerID="27b276c29da6f8840d94182458f6c48e93963c1187d3429537c7e563a01524c2"
Jul 17 05:45:50.002887 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:50.002870    2562 scope.go:117] "RemoveContainer" containerID="4d76cd160f76415e079135eda1f1309d69825c2099a7df0deb78d976f46560fd"
Jul 17 05:45:50.011110 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:50.011090    2562 scope.go:117] "RemoveContainer" containerID="27b276c29da6f8840d94182458f6c48e93963c1187d3429537c7e563a01524c2"
Jul 17 05:45:50.011415 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:45:50.011376    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27b276c29da6f8840d94182458f6c48e93963c1187d3429537c7e563a01524c2\": container with ID starting with 27b276c29da6f8840d94182458f6c48e93963c1187d3429537c7e563a01524c2 not found: ID does not exist" containerID="27b276c29da6f8840d94182458f6c48e93963c1187d3429537c7e563a01524c2"
Jul 17 05:45:50.011475 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:50.011422    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27b276c29da6f8840d94182458f6c48e93963c1187d3429537c7e563a01524c2"} err="failed to get container status \"27b276c29da6f8840d94182458f6c48e93963c1187d3429537c7e563a01524c2\": rpc error: code = NotFound desc = could not find container \"27b276c29da6f8840d94182458f6c48e93963c1187d3429537c7e563a01524c2\": container with ID starting with 27b276c29da6f8840d94182458f6c48e93963c1187d3429537c7e563a01524c2 not found: ID does not exist"
Jul 17 05:45:50.011475 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:50.011439    2562 scope.go:117] "RemoveContainer" containerID="4d76cd160f76415e079135eda1f1309d69825c2099a7df0deb78d976f46560fd"
Jul 17 05:45:50.011727 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:45:50.011694    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d76cd160f76415e079135eda1f1309d69825c2099a7df0deb78d976f46560fd\": container with ID starting with 4d76cd160f76415e079135eda1f1309d69825c2099a7df0deb78d976f46560fd not found: ID does not exist" containerID="4d76cd160f76415e079135eda1f1309d69825c2099a7df0deb78d976f46560fd"
Jul 17 05:45:50.011826 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:50.011731    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d76cd160f76415e079135eda1f1309d69825c2099a7df0deb78d976f46560fd"} err="failed to get container status \"4d76cd160f76415e079135eda1f1309d69825c2099a7df0deb78d976f46560fd\": rpc error: code = NotFound desc = could not find container \"4d76cd160f76415e079135eda1f1309d69825c2099a7df0deb78d976f46560fd\": container with ID starting with 4d76cd160f76415e079135eda1f1309d69825c2099a7df0deb78d976f46560fd not found: ID does not exist"
Jul 17 05:45:50.015459 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:50.015433    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6"]
Jul 17 05:45:50.018492 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:50.018471    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-560f5b-openshift-default-6d9449f955-98jq6"]
Jul 17 05:45:50.885010 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:50.884945    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0357b315-a3ca-4c24-b0e5-51ee48d89983" path="/var/lib/kubelet/pods/0357b315-a3ca-4c24-b0e5-51ee48d89983/volumes"
Jul 17 05:45:55.342928 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:55.342890    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-823cda-29737785-8bnxb"]
Jul 17 05:45:55.345788 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:55.345763    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-823cda-29737785-8bnxb"]
Jul 17 05:45:56.884325 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:45:56.884292    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="356830d7-dbe1-4b96-b025-232e35a1a70b" path="/var/lib/kubelet/pods/356830d7-dbe1-4b96-b025-232e35a1a70b/volumes"
Jul 17 05:46:10.656286 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:10.656242    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx"]
Jul 17 05:46:10.656907 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:10.656487    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx" podUID="99aa5e8d-f861-41d8-a8db-e764ae921284" containerName="istio-proxy" containerID="cri-o://45bbd84517de52c01d9bbf8a431e080ecae77d21dce15d577a056fc6cfe58863" gracePeriod=30
Jul 17 05:46:15.920304 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:15.920242    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx"
Jul 17 05:46:15.972469 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:15.972434    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99aa5e8d-f861-41d8-a8db-e764ae921284-workload-socket\") pod \"99aa5e8d-f861-41d8-a8db-e764ae921284\" (UID: \"99aa5e8d-f861-41d8-a8db-e764ae921284\") "
Jul 17 05:46:15.972469 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:15.972474    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99aa5e8d-f861-41d8-a8db-e764ae921284-istio-token\") pod \"99aa5e8d-f861-41d8-a8db-e764ae921284\" (UID: \"99aa5e8d-f861-41d8-a8db-e764ae921284\") "
Jul 17 05:46:15.972721 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:15.972494    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99aa5e8d-f861-41d8-a8db-e764ae921284-workload-certs\") pod \"99aa5e8d-f861-41d8-a8db-e764ae921284\" (UID: \"99aa5e8d-f861-41d8-a8db-e764ae921284\") "
Jul 17 05:46:15.972721 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:15.972525    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9lwmw\" (UniqueName: \"kubernetes.io/projected/99aa5e8d-f861-41d8-a8db-e764ae921284-kube-api-access-9lwmw\") pod \"99aa5e8d-f861-41d8-a8db-e764ae921284\" (UID: \"99aa5e8d-f861-41d8-a8db-e764ae921284\") "
Jul 17 05:46:15.972721 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:15.972626    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99aa5e8d-f861-41d8-a8db-e764ae921284-istio-envoy\") pod \"99aa5e8d-f861-41d8-a8db-e764ae921284\" (UID: \"99aa5e8d-f861-41d8-a8db-e764ae921284\") "
Jul 17 05:46:15.972721 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:15.972697    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99aa5e8d-f861-41d8-a8db-e764ae921284-credential-socket\") pod \"99aa5e8d-f861-41d8-a8db-e764ae921284\" (UID: \"99aa5e8d-f861-41d8-a8db-e764ae921284\") "
Jul 17 05:46:15.972875 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:15.972729    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99aa5e8d-f861-41d8-a8db-e764ae921284-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "99aa5e8d-f861-41d8-a8db-e764ae921284" (UID: "99aa5e8d-f861-41d8-a8db-e764ae921284"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:46:15.972875 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:15.972793    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99aa5e8d-f861-41d8-a8db-e764ae921284-istio-podinfo\") pod \"99aa5e8d-f861-41d8-a8db-e764ae921284\" (UID: \"99aa5e8d-f861-41d8-a8db-e764ae921284\") "
Jul 17 05:46:15.972875 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:15.972826    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99aa5e8d-f861-41d8-a8db-e764ae921284-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "99aa5e8d-f861-41d8-a8db-e764ae921284" (UID: "99aa5e8d-f861-41d8-a8db-e764ae921284"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:46:15.972875 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:15.972840    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99aa5e8d-f861-41d8-a8db-e764ae921284-istiod-ca-cert\") pod \"99aa5e8d-f861-41d8-a8db-e764ae921284\" (UID: \"99aa5e8d-f861-41d8-a8db-e764ae921284\") "
Jul 17 05:46:15.973122 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:15.972883    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99aa5e8d-f861-41d8-a8db-e764ae921284-istio-data\") pod \"99aa5e8d-f861-41d8-a8db-e764ae921284\" (UID: \"99aa5e8d-f861-41d8-a8db-e764ae921284\") "
Jul 17 05:46:15.973122 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:15.972951    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99aa5e8d-f861-41d8-a8db-e764ae921284-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "99aa5e8d-f861-41d8-a8db-e764ae921284" (UID: "99aa5e8d-f861-41d8-a8db-e764ae921284"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:46:15.973245 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:15.973220    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99aa5e8d-f861-41d8-a8db-e764ae921284-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "99aa5e8d-f861-41d8-a8db-e764ae921284" (UID: "99aa5e8d-f861-41d8-a8db-e764ae921284"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:46:15.973322 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:15.973250    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99aa5e8d-f861-41d8-a8db-e764ae921284-workload-certs\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:46:15.973322 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:15.973268    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99aa5e8d-f861-41d8-a8db-e764ae921284-credential-socket\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:46:15.973322 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:15.973281    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99aa5e8d-f861-41d8-a8db-e764ae921284-workload-socket\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:46:15.973736 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:15.973711    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99aa5e8d-f861-41d8-a8db-e764ae921284-istio-data" (OuterVolumeSpecName: "istio-data") pod "99aa5e8d-f861-41d8-a8db-e764ae921284" (UID: "99aa5e8d-f861-41d8-a8db-e764ae921284"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:46:15.974897 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:15.974870    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99aa5e8d-f861-41d8-a8db-e764ae921284-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "99aa5e8d-f861-41d8-a8db-e764ae921284" (UID: "99aa5e8d-f861-41d8-a8db-e764ae921284"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 05:46:15.975075 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:15.975051    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99aa5e8d-f861-41d8-a8db-e764ae921284-istio-token" (OuterVolumeSpecName: "istio-token") pod "99aa5e8d-f861-41d8-a8db-e764ae921284" (UID: "99aa5e8d-f861-41d8-a8db-e764ae921284"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:46:15.975328 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:15.975313    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99aa5e8d-f861-41d8-a8db-e764ae921284-kube-api-access-9lwmw" (OuterVolumeSpecName: "kube-api-access-9lwmw") pod "99aa5e8d-f861-41d8-a8db-e764ae921284" (UID: "99aa5e8d-f861-41d8-a8db-e764ae921284"). InnerVolumeSpecName "kube-api-access-9lwmw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 05:46:15.975372 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:15.975348    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/99aa5e8d-f861-41d8-a8db-e764ae921284-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "99aa5e8d-f861-41d8-a8db-e764ae921284" (UID: "99aa5e8d-f861-41d8-a8db-e764ae921284"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 05:46:16.074035 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:16.073995    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99aa5e8d-f861-41d8-a8db-e764ae921284-istio-podinfo\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:46:16.074035 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:16.074031    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99aa5e8d-f861-41d8-a8db-e764ae921284-istiod-ca-cert\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:46:16.074035 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:16.074043    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99aa5e8d-f861-41d8-a8db-e764ae921284-istio-data\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:46:16.074293 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:16.074055    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99aa5e8d-f861-41d8-a8db-e764ae921284-istio-token\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:46:16.074293 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:16.074067    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9lwmw\" (UniqueName: \"kubernetes.io/projected/99aa5e8d-f861-41d8-a8db-e764ae921284-kube-api-access-9lwmw\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:46:16.074293 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:16.074079    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99aa5e8d-f861-41d8-a8db-e764ae921284-istio-envoy\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:46:16.134216 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:16.134175    2562 generic.go:356] "Generic (PLEG): container finished" podID="99aa5e8d-f861-41d8-a8db-e764ae921284" containerID="45bbd84517de52c01d9bbf8a431e080ecae77d21dce15d577a056fc6cfe58863" exitCode=0
Jul 17 05:46:16.134427 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:16.134236    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx"
Jul 17 05:46:16.134427 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:16.134269    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx" event={"ID":"99aa5e8d-f861-41d8-a8db-e764ae921284","Type":"ContainerDied","Data":"45bbd84517de52c01d9bbf8a431e080ecae77d21dce15d577a056fc6cfe58863"}
Jul 17 05:46:16.134427 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:16.134330    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx" event={"ID":"99aa5e8d-f861-41d8-a8db-e764ae921284","Type":"ContainerDied","Data":"49f2eaeef79b2d72534234702ea0bbfc308743e6ef3453532a4a8cc6ae5a1beb"}
Jul 17 05:46:16.134427 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:16.134358    2562 scope.go:117] "RemoveContainer" containerID="45bbd84517de52c01d9bbf8a431e080ecae77d21dce15d577a056fc6cfe58863"
Jul 17 05:46:16.145063 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:16.145041    2562 scope.go:117] "RemoveContainer" containerID="ebc95b438fd814482b1975dc9ffdb71570d5dda8ee21bc833cedaf8c646ba3f0"
Jul 17 05:46:16.153225 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:16.153207    2562 scope.go:117] "RemoveContainer" containerID="45bbd84517de52c01d9bbf8a431e080ecae77d21dce15d577a056fc6cfe58863"
Jul 17 05:46:16.153512 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:46:16.153484    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45bbd84517de52c01d9bbf8a431e080ecae77d21dce15d577a056fc6cfe58863\": container with ID starting with 45bbd84517de52c01d9bbf8a431e080ecae77d21dce15d577a056fc6cfe58863 not found: ID does not exist" containerID="45bbd84517de52c01d9bbf8a431e080ecae77d21dce15d577a056fc6cfe58863"
Jul 17 05:46:16.153691 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:16.153546    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45bbd84517de52c01d9bbf8a431e080ecae77d21dce15d577a056fc6cfe58863"} err="failed to get container status \"45bbd84517de52c01d9bbf8a431e080ecae77d21dce15d577a056fc6cfe58863\": rpc error: code = NotFound desc = could not find container \"45bbd84517de52c01d9bbf8a431e080ecae77d21dce15d577a056fc6cfe58863\": container with ID starting with 45bbd84517de52c01d9bbf8a431e080ecae77d21dce15d577a056fc6cfe58863 not found: ID does not exist"
Jul 17 05:46:16.153771 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:16.153578    2562 scope.go:117] "RemoveContainer" containerID="ebc95b438fd814482b1975dc9ffdb71570d5dda8ee21bc833cedaf8c646ba3f0"
Jul 17 05:46:16.154098 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:46:16.154070    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebc95b438fd814482b1975dc9ffdb71570d5dda8ee21bc833cedaf8c646ba3f0\": container with ID starting with ebc95b438fd814482b1975dc9ffdb71570d5dda8ee21bc833cedaf8c646ba3f0 not found: ID does not exist" containerID="ebc95b438fd814482b1975dc9ffdb71570d5dda8ee21bc833cedaf8c646ba3f0"
Jul 17 05:46:16.154190 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:16.154107    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebc95b438fd814482b1975dc9ffdb71570d5dda8ee21bc833cedaf8c646ba3f0"} err="failed to get container status \"ebc95b438fd814482b1975dc9ffdb71570d5dda8ee21bc833cedaf8c646ba3f0\": rpc error: code = NotFound desc = could not find container \"ebc95b438fd814482b1975dc9ffdb71570d5dda8ee21bc833cedaf8c646ba3f0\": container with ID starting with ebc95b438fd814482b1975dc9ffdb71570d5dda8ee21bc833cedaf8c646ba3f0 not found: ID does not exist"
Jul 17 05:46:16.156375 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:16.156355    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx"]
Jul 17 05:46:16.160154 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:16.160133    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-823cda-openshift-default-cc9979b79-t9czx"]
Jul 17 05:46:16.885416 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:16.885380    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99aa5e8d-f861-41d8-a8db-e764ae921284" path="/var/lib/kubelet/pods/99aa5e8d-f861-41d8-a8db-e764ae921284/volumes"
Jul 17 05:46:17.113760 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:17.113702    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-859ff64978-dq8x9_8f3b63a1-b0a2-42d4-ab11-11ad32132d44/manager/0.log"
Jul 17 05:46:17.601299 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:17.601271    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d857c78d5-ft2q8_a667f213-da2d-4290-b67c-700500da24b0/copy-manifests/0.log"
Jul 17 05:46:17.609953 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:17.609915    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d857c78d5-ft2q8_a667f213-da2d-4290-b67c-700500da24b0/manager/0.log"
Jul 17 05:46:17.782989 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:17.781148    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-rdh8m"]
Jul 17 05:46:17.782989 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:17.782053    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99aa5e8d-f861-41d8-a8db-e764ae921284" containerName="istio-proxy"
Jul 17 05:46:17.782989 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:17.782074    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="99aa5e8d-f861-41d8-a8db-e764ae921284" containerName="istio-proxy"
Jul 17 05:46:17.782989 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:17.782105    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99aa5e8d-f861-41d8-a8db-e764ae921284" containerName="istio-proxy"
Jul 17 05:46:17.782989 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:17.782114    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="99aa5e8d-f861-41d8-a8db-e764ae921284" containerName="istio-proxy"
Jul 17 05:46:17.782989 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:17.782136    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0357b315-a3ca-4c24-b0e5-51ee48d89983" containerName="istio-proxy"
Jul 17 05:46:17.782989 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:17.782145    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0357b315-a3ca-4c24-b0e5-51ee48d89983" containerName="istio-proxy"
Jul 17 05:46:17.782989 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:17.782175    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0357b315-a3ca-4c24-b0e5-51ee48d89983" containerName="istio-proxy"
Jul 17 05:46:17.782989 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:17.782184    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0357b315-a3ca-4c24-b0e5-51ee48d89983" containerName="istio-proxy"
Jul 17 05:46:17.782989 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:17.782202    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4132650c-801d-42c3-9c48-93ce44c8351e" containerName="revoke-keys"
Jul 17 05:46:17.782989 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:17.782210    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4132650c-801d-42c3-9c48-93ce44c8351e" containerName="revoke-keys"
Jul 17 05:46:17.782989 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:17.782250    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4132650c-801d-42c3-9c48-93ce44c8351e" containerName="revoke-keys"
Jul 17 05:46:17.782989 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:17.782259    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4132650c-801d-42c3-9c48-93ce44c8351e" containerName="revoke-keys"
Jul 17 05:46:17.782989 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:17.782295    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4132650c-801d-42c3-9c48-93ce44c8351e" containerName="revoke-keys"
Jul 17 05:46:17.782989 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:17.782303    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4132650c-801d-42c3-9c48-93ce44c8351e" containerName="revoke-keys"
Jul 17 05:46:17.782989 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:17.782475    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="99aa5e8d-f861-41d8-a8db-e764ae921284" containerName="istio-proxy"
Jul 17 05:46:17.782989 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:17.782492    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4132650c-801d-42c3-9c48-93ce44c8351e" containerName="revoke-keys"
Jul 17 05:46:17.782989 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:17.782509    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4132650c-801d-42c3-9c48-93ce44c8351e" containerName="revoke-keys"
Jul 17 05:46:17.782989 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:17.782519    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="99aa5e8d-f861-41d8-a8db-e764ae921284" containerName="istio-proxy"
Jul 17 05:46:17.782989 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:17.782539    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0357b315-a3ca-4c24-b0e5-51ee48d89983" containerName="istio-proxy"
Jul 17 05:46:17.782989 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:17.782547    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0357b315-a3ca-4c24-b0e5-51ee48d89983" containerName="istio-proxy"
Jul 17 05:46:17.789689 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:17.789643    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-rdh8m"]
Jul 17 05:46:17.789855 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:17.789759    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-rdh8m"
Jul 17 05:46:17.792572 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:17.792547    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qh8zj\""
Jul 17 05:46:17.889811 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:17.889715    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/806ece69-9272-4c82-bb09-cd9f33d884c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9eb8486e-rdh8m\" (UID: \"806ece69-9272-4c82-bb09-cd9f33d884c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-rdh8m"
Jul 17 05:46:17.991146 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:17.991091    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/806ece69-9272-4c82-bb09-cd9f33d884c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9eb8486e-rdh8m\" (UID: \"806ece69-9272-4c82-bb09-cd9f33d884c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-rdh8m"
Jul 17 05:46:17.991719 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:17.991699    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/806ece69-9272-4c82-bb09-cd9f33d884c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9eb8486e-rdh8m\" (UID: \"806ece69-9272-4c82-bb09-cd9f33d884c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-rdh8m"
Jul 17 05:46:18.100699 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:18.100670    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-rdh8m"
Jul 17 05:46:18.107942 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:18.107836    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-7954f7d7ff-j7k45_8c259023-8aaa-48ef-b9b2-6818d76290eb/maas-api/0.log"
Jul 17 05:46:18.231394 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:18.231367    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29737755-6zrtq_c9f3d10c-1add-4371-984d-56d64050ca2e/cleanup/0.log"
Jul 17 05:46:18.239857 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:18.239831    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-rdh8m"]
Jul 17 05:46:18.241810 ip-10-0-132-19 kubenswrapper[2562]: W0717 05:46:18.241778    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod806ece69_9272_4c82_bb09_cd9f33d884c3.slice/crio-1c4af5db1af1c5cb1fe2344c6895285ecbd17256823dfdae97029b6a9093f5f3 WatchSource:0}: Error finding container 1c4af5db1af1c5cb1fe2344c6895285ecbd17256823dfdae97029b6a9093f5f3: Status 404 returned error can't find the container with id 1c4af5db1af1c5cb1fe2344c6895285ecbd17256823dfdae97029b6a9093f5f3
Jul 17 05:46:18.451501 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:18.451424    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29737785-kdjmj_a8817732-a28b-466e-9c7d-c68a72050030/cleanup/0.log"
Jul 17 05:46:19.152729 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:19.152638    2562 generic.go:356] "Generic (PLEG): container finished" podID="806ece69-9272-4c82-bb09-cd9f33d884c3" containerID="80a9812621812b7c4411678accd2ea48460205773cc320053bc48e0151ce7fa9" exitCode=6
Jul 17 05:46:19.152729 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:19.152680    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-rdh8m" event={"ID":"806ece69-9272-4c82-bb09-cd9f33d884c3","Type":"ContainerDied","Data":"80a9812621812b7c4411678accd2ea48460205773cc320053bc48e0151ce7fa9"}
Jul 17 05:46:19.152914 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:19.152732    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-rdh8m" event={"ID":"806ece69-9272-4c82-bb09-cd9f33d884c3","Type":"ContainerStarted","Data":"1c4af5db1af1c5cb1fe2344c6895285ecbd17256823dfdae97029b6a9093f5f3"}
Jul 17 05:46:19.152954 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:19.152940    2562 scope.go:117] "RemoveContainer" containerID="80a9812621812b7c4411678accd2ea48460205773cc320053bc48e0151ce7fa9"
Jul 17 05:46:19.677249 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:19.677204    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-zbx4j_fd4f9eb2-a49c-46c6-909e-64392cc37e39/postgres/0.log"
Jul 17 05:46:20.160859 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:20.160822    2562 generic.go:356] "Generic (PLEG): container finished" podID="806ece69-9272-4c82-bb09-cd9f33d884c3" containerID="dff9c6580193185d6b5afdf20a96660ee8f5c04176d5affd6dde23f52834b3f4" exitCode=6
Jul 17 05:46:20.161090 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:20.160871    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-rdh8m" event={"ID":"806ece69-9272-4c82-bb09-cd9f33d884c3","Type":"ContainerDied","Data":"dff9c6580193185d6b5afdf20a96660ee8f5c04176d5affd6dde23f52834b3f4"}
Jul 17 05:46:20.161090 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:20.160984    2562 scope.go:117] "RemoveContainer" containerID="80a9812621812b7c4411678accd2ea48460205773cc320053bc48e0151ce7fa9"
Jul 17 05:46:20.161218 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:20.161154    2562 scope.go:117] "RemoveContainer" containerID="dff9c6580193185d6b5afdf20a96660ee8f5c04176d5affd6dde23f52834b3f4"
Jul 17 05:46:20.161402 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:46:20.161380    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-9eb8486e-rdh8m_odh-ai-gateway-infra(806ece69-9272-4c82-bb09-cd9f33d884c3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-rdh8m" podUID="806ece69-9272-4c82-bb09-cd9f33d884c3"
Jul 17 05:46:21.167511 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:21.167479    2562 scope.go:117] "RemoveContainer" containerID="dff9c6580193185d6b5afdf20a96660ee8f5c04176d5affd6dde23f52834b3f4"
Jul 17 05:46:21.167894 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:46:21.167686    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-9eb8486e-rdh8m_odh-ai-gateway-infra(806ece69-9272-4c82-bb09-cd9f33d884c3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-rdh8m" podUID="806ece69-9272-4c82-bb09-cd9f33d884c3"
Jul 17 05:46:22.295028 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:22.294989    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-9tb57_e0a7089b-2cde-439a-8cd8-187338f02f18/discovery/0.log"
Jul 17 05:46:22.657725 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:22.657681    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-k2snx_162f43aa-9c5c-4148-8f41-4c1f43c9d7a9/istio-proxy/1.log"
Jul 17 05:46:22.781301 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:22.781271    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-qc8sv_fc5f1ecb-28e8-49ce-89d7-b841f10e04b4/payload-pre-processing/0.log"
Jul 17 05:46:23.356553 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:23.356522    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-vkz7b_28ddd366-d845-4f5a-984c-08e9805563a1/main/0.log"
Jul 17 05:46:23.468112 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:23.468073    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-bjn2w_5e694bf8-4bfc-49c4-82e1-6b46739868cc/main/0.log"
Jul 17 05:46:29.737220 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:29.737178    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-r6hhr"]
Jul 17 05:46:29.737843 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:29.737823    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4132650c-801d-42c3-9c48-93ce44c8351e" containerName="revoke-keys"
Jul 17 05:46:29.740883 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:29.740860    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-r6hhr"
Jul 17 05:46:29.752938 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:29.752914    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-r6hhr"]
Jul 17 05:46:29.806122 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:29.806088    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e228b24d-8a8d-4c35-a2bd-52392292acbf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-c8a383ce-r6hhr\" (UID: \"e228b24d-8a8d-4c35-a2bd-52392292acbf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-r6hhr"
Jul 17 05:46:29.907115 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:29.907075    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e228b24d-8a8d-4c35-a2bd-52392292acbf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-c8a383ce-r6hhr\" (UID: \"e228b24d-8a8d-4c35-a2bd-52392292acbf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-r6hhr"
Jul 17 05:46:29.907729 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:29.907706    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e228b24d-8a8d-4c35-a2bd-52392292acbf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-c8a383ce-r6hhr\" (UID: \"e228b24d-8a8d-4c35-a2bd-52392292acbf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-r6hhr"
Jul 17 05:46:30.051365 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:30.051258    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-r6hhr"
Jul 17 05:46:30.194945 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:30.194915    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-r6hhr"]
Jul 17 05:46:30.197686 ip-10-0-132-19 kubenswrapper[2562]: W0717 05:46:30.197651    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode228b24d_8a8d_4c35_a2bd_52392292acbf.slice/crio-6034ce92c5a2ec0260cf054c984fa3abd4520603fadb96db9e29935211efccdd WatchSource:0}: Error finding container 6034ce92c5a2ec0260cf054c984fa3abd4520603fadb96db9e29935211efccdd: Status 404 returned error can't find the container with id 6034ce92c5a2ec0260cf054c984fa3abd4520603fadb96db9e29935211efccdd
Jul 17 05:46:30.212434 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:30.212405    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-r6hhr" event={"ID":"e228b24d-8a8d-4c35-a2bd-52392292acbf","Type":"ContainerStarted","Data":"6034ce92c5a2ec0260cf054c984fa3abd4520603fadb96db9e29935211efccdd"}
Jul 17 05:46:31.218762 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:31.218729    2562 generic.go:356] "Generic (PLEG): container finished" podID="e228b24d-8a8d-4c35-a2bd-52392292acbf" containerID="eeedaefd9e8facf62f5acdb04585bca516608a1a6a39054cf34919e56fd76b4f" exitCode=6
Jul 17 05:46:31.219233 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:31.218825    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-r6hhr" event={"ID":"e228b24d-8a8d-4c35-a2bd-52392292acbf","Type":"ContainerDied","Data":"eeedaefd9e8facf62f5acdb04585bca516608a1a6a39054cf34919e56fd76b4f"}
Jul 17 05:46:31.219233 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:31.218985    2562 scope.go:117] "RemoveContainer" containerID="eeedaefd9e8facf62f5acdb04585bca516608a1a6a39054cf34919e56fd76b4f"
Jul 17 05:46:32.226683 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:32.226641    2562 generic.go:356] "Generic (PLEG): container finished" podID="e228b24d-8a8d-4c35-a2bd-52392292acbf" containerID="8d531b7ce2a5a319ed5b059865bc3c192f727bb125a4e24a96c76e5e3d73f49b" exitCode=6
Jul 17 05:46:32.227143 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:32.226733    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-r6hhr" event={"ID":"e228b24d-8a8d-4c35-a2bd-52392292acbf","Type":"ContainerDied","Data":"8d531b7ce2a5a319ed5b059865bc3c192f727bb125a4e24a96c76e5e3d73f49b"}
Jul 17 05:46:32.227143 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:32.226800    2562 scope.go:117] "RemoveContainer" containerID="eeedaefd9e8facf62f5acdb04585bca516608a1a6a39054cf34919e56fd76b4f"
Jul 17 05:46:32.227143 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:32.226922    2562 scope.go:117] "RemoveContainer" containerID="8d531b7ce2a5a319ed5b059865bc3c192f727bb125a4e24a96c76e5e3d73f49b"
Jul 17 05:46:32.227300 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:46:32.227196    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-c8a383ce-r6hhr_odh-ai-gateway-infra(e228b24d-8a8d-4c35-a2bd-52392292acbf)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-r6hhr" podUID="e228b24d-8a8d-4c35-a2bd-52392292acbf"
Jul 17 05:46:32.269543 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:32.269506    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6bkt5_237abccc-c19c-4fca-be94-d3013f3c2e86/global-pull-secret-syncer/0.log"
Jul 17 05:46:32.476742 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:32.476660    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-nbdgc_da4d7ee5-31c8-465a-b5bd-5127ac635d08/konnectivity-agent/0.log"
Jul 17 05:46:32.526856 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:32.526822    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-19.ec2.internal_02dd81e7d773510fb8d800d8518d7fdd/haproxy/0.log"
Jul 17 05:46:33.233447 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:33.233422    2562 scope.go:117] "RemoveContainer" containerID="8d531b7ce2a5a319ed5b059865bc3c192f727bb125a4e24a96c76e5e3d73f49b"
Jul 17 05:46:33.233840 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:46:33.233629    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-c8a383ce-r6hhr_odh-ai-gateway-infra(e228b24d-8a8d-4c35-a2bd-52392292acbf)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-r6hhr" podUID="e228b24d-8a8d-4c35-a2bd-52392292acbf"
Jul 17 05:46:33.878347 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:33.878312    2562 scope.go:117] "RemoveContainer" containerID="dff9c6580193185d6b5afdf20a96660ee8f5c04176d5affd6dde23f52834b3f4"
Jul 17 05:46:34.240578 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:34.240546    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-rdh8m" event={"ID":"806ece69-9272-4c82-bb09-cd9f33d884c3","Type":"ContainerStarted","Data":"69a7ff8a91b691cc99a5f299278a5e04a37f3039a599cd5cbcd85d89701dcb03"}
Jul 17 05:46:34.256331 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:34.256276    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-rdh8m" podStartSLOduration=17.256256005 podStartE2EDuration="17.256256005s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:46:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:46:34.255035903 +0000 UTC m=+3381.938629578" watchObservedRunningTime="2026-07-17 05:46:34.256256005 +0000 UTC m=+3381.939849665"
Jul 17 05:46:35.120698 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:35.120663    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-8x5rn"]
Jul 17 05:46:35.124263 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:35.124247    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-8x5rn"
Jul 17 05:46:35.131189 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:35.131160    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-8x5rn"]
Jul 17 05:46:35.247298 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:35.247264    2562 generic.go:356] "Generic (PLEG): container finished" podID="806ece69-9272-4c82-bb09-cd9f33d884c3" containerID="69a7ff8a91b691cc99a5f299278a5e04a37f3039a599cd5cbcd85d89701dcb03" exitCode=6
Jul 17 05:46:35.247690 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:35.247333    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-rdh8m" event={"ID":"806ece69-9272-4c82-bb09-cd9f33d884c3","Type":"ContainerDied","Data":"69a7ff8a91b691cc99a5f299278a5e04a37f3039a599cd5cbcd85d89701dcb03"}
Jul 17 05:46:35.247690 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:35.247396    2562 scope.go:117] "RemoveContainer" containerID="dff9c6580193185d6b5afdf20a96660ee8f5c04176d5affd6dde23f52834b3f4"
Jul 17 05:46:35.247690 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:35.247617    2562 scope.go:117] "RemoveContainer" containerID="69a7ff8a91b691cc99a5f299278a5e04a37f3039a599cd5cbcd85d89701dcb03"
Jul 17 05:46:35.247888 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:46:35.247863    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-9eb8486e-rdh8m_odh-ai-gateway-infra(806ece69-9272-4c82-bb09-cd9f33d884c3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-rdh8m" podUID="806ece69-9272-4c82-bb09-cd9f33d884c3"
Jul 17 05:46:35.259007 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:35.258983    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c2bb0e6-2065-457e-96e8-27a6170fba95-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-bf0ad9db-8x5rn\" (UID: \"2c2bb0e6-2065-457e-96e8-27a6170fba95\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-8x5rn"
Jul 17 05:46:35.267108 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:35.267084    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-rdh8m"]
Jul 17 05:46:35.360179 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:35.360145    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c2bb0e6-2065-457e-96e8-27a6170fba95-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-bf0ad9db-8x5rn\" (UID: \"2c2bb0e6-2065-457e-96e8-27a6170fba95\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-8x5rn"
Jul 17 05:46:35.360719 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:35.360701    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c2bb0e6-2065-457e-96e8-27a6170fba95-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-bf0ad9db-8x5rn\" (UID: \"2c2bb0e6-2065-457e-96e8-27a6170fba95\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-8x5rn"
Jul 17 05:46:35.436448 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:35.436362    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-8x5rn"
Jul 17 05:46:35.572947 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:35.572922    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-8x5rn"]
Jul 17 05:46:35.574860 ip-10-0-132-19 kubenswrapper[2562]: W0717 05:46:35.574832    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c2bb0e6_2065_457e_96e8_27a6170fba95.slice/crio-d72ee8f44a4ab97c2c726462b0ef28197d932c9d3ac12ae8163f9b4f56599bbb WatchSource:0}: Error finding container d72ee8f44a4ab97c2c726462b0ef28197d932c9d3ac12ae8163f9b4f56599bbb: Status 404 returned error can't find the container with id d72ee8f44a4ab97c2c726462b0ef28197d932c9d3ac12ae8163f9b4f56599bbb
Jul 17 05:46:36.254039 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:36.253999    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-8x5rn" event={"ID":"2c2bb0e6-2065-457e-96e8-27a6170fba95","Type":"ContainerStarted","Data":"19ed07102420c58bdea268fb68bac9079987013061294879a4d724ea888ada66"}
Jul 17 05:46:36.254542 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:36.254062    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-8x5rn" event={"ID":"2c2bb0e6-2065-457e-96e8-27a6170fba95","Type":"ContainerStarted","Data":"d72ee8f44a4ab97c2c726462b0ef28197d932c9d3ac12ae8163f9b4f56599bbb"}
Jul 17 05:46:36.270256 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:36.270196    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-8x5rn" podStartSLOduration=1.2701789030000001 podStartE2EDuration="1.270178903s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:46:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:46:36.268215618 +0000 UTC m=+3383.951809276" watchObservedRunningTime="2026-07-17 05:46:36.270178903 +0000 UTC m=+3383.953772559"
Jul 17 05:46:36.382198 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:36.382175    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-rdh8m"
Jul 17 05:46:36.471362 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:36.471327    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/806ece69-9272-4c82-bb09-cd9f33d884c3-maas-api-service-ca\") pod \"806ece69-9272-4c82-bb09-cd9f33d884c3\" (UID: \"806ece69-9272-4c82-bb09-cd9f33d884c3\") "
Jul 17 05:46:36.471693 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:36.471670    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/806ece69-9272-4c82-bb09-cd9f33d884c3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "806ece69-9272-4c82-bb09-cd9f33d884c3" (UID: "806ece69-9272-4c82-bb09-cd9f33d884c3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:46:36.572940 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:36.572889    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/806ece69-9272-4c82-bb09-cd9f33d884c3-maas-api-service-ca\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:46:37.262801 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:37.262712    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-rdh8m"
Jul 17 05:46:37.262801 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:37.262732    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-rdh8m" event={"ID":"806ece69-9272-4c82-bb09-cd9f33d884c3","Type":"ContainerDied","Data":"1c4af5db1af1c5cb1fe2344c6895285ecbd17256823dfdae97029b6a9093f5f3"}
Jul 17 05:46:37.262801 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:37.262798    2562 scope.go:117] "RemoveContainer" containerID="69a7ff8a91b691cc99a5f299278a5e04a37f3039a599cd5cbcd85d89701dcb03"
Jul 17 05:46:37.264657 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:37.264614    2562 generic.go:356] "Generic (PLEG): container finished" podID="2c2bb0e6-2065-457e-96e8-27a6170fba95" containerID="19ed07102420c58bdea268fb68bac9079987013061294879a4d724ea888ada66" exitCode=6
Jul 17 05:46:37.264777 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:37.264651    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-8x5rn" event={"ID":"2c2bb0e6-2065-457e-96e8-27a6170fba95","Type":"ContainerDied","Data":"19ed07102420c58bdea268fb68bac9079987013061294879a4d724ea888ada66"}
Jul 17 05:46:37.264941 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:37.264921    2562 scope.go:117] "RemoveContainer" containerID="19ed07102420c58bdea268fb68bac9079987013061294879a4d724ea888ada66"
Jul 17 05:46:37.282923 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:37.282897    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-rdh8m"]
Jul 17 05:46:37.287453 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:37.287428    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9eb8486e-rdh8m"]
Jul 17 05:46:38.271525 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:38.271441    2562 generic.go:356] "Generic (PLEG): container finished" podID="2c2bb0e6-2065-457e-96e8-27a6170fba95" containerID="ba09c9c6da293e81b851b632a1b82fd620c4d2e08de2f1da9e196724cbb72687" exitCode=6
Jul 17 05:46:38.271928 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:38.271527    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-8x5rn" event={"ID":"2c2bb0e6-2065-457e-96e8-27a6170fba95","Type":"ContainerDied","Data":"ba09c9c6da293e81b851b632a1b82fd620c4d2e08de2f1da9e196724cbb72687"}
Jul 17 05:46:38.271928 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:38.271593    2562 scope.go:117] "RemoveContainer" containerID="19ed07102420c58bdea268fb68bac9079987013061294879a4d724ea888ada66"
Jul 17 05:46:38.271928 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:38.271853    2562 scope.go:117] "RemoveContainer" containerID="ba09c9c6da293e81b851b632a1b82fd620c4d2e08de2f1da9e196724cbb72687"
Jul 17 05:46:38.272153 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:46:38.272127    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-bf0ad9db-8x5rn_odh-ai-gateway-infra(2c2bb0e6-2065-457e-96e8-27a6170fba95)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-8x5rn" podUID="2c2bb0e6-2065-457e-96e8-27a6170fba95"
Jul 17 05:46:38.884083 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:38.884046    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="806ece69-9272-4c82-bb09-cd9f33d884c3" path="/var/lib/kubelet/pods/806ece69-9272-4c82-bb09-cd9f33d884c3/volumes"
Jul 17 05:46:39.279889 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:39.279861    2562 scope.go:117] "RemoveContainer" containerID="ba09c9c6da293e81b851b632a1b82fd620c4d2e08de2f1da9e196724cbb72687"
Jul 17 05:46:39.280320 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:46:39.280127    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-bf0ad9db-8x5rn_odh-ai-gateway-infra(2c2bb0e6-2065-457e-96e8-27a6170fba95)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-8x5rn" podUID="2c2bb0e6-2065-457e-96e8-27a6170fba95"
Jul 17 05:46:39.342682 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:39.342653    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2975ade8-a4ab-4bec-9781-64655a501d21/alertmanager/0.log"
Jul 17 05:46:39.362483 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:39.362450    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2975ade8-a4ab-4bec-9781-64655a501d21/config-reloader/0.log"
Jul 17 05:46:39.386397 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:39.386353    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2975ade8-a4ab-4bec-9781-64655a501d21/kube-rbac-proxy-web/0.log"
Jul 17 05:46:39.408063 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:39.407930    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2975ade8-a4ab-4bec-9781-64655a501d21/kube-rbac-proxy/0.log"
Jul 17 05:46:39.430368 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:39.430332    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2975ade8-a4ab-4bec-9781-64655a501d21/kube-rbac-proxy-metric/0.log"
Jul 17 05:46:39.454145 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:39.454120    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2975ade8-a4ab-4bec-9781-64655a501d21/prom-label-proxy/0.log"
Jul 17 05:46:39.474849 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:39.474823    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2975ade8-a4ab-4bec-9781-64655a501d21/init-config-reloader/0.log"
Jul 17 05:46:39.567466 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:39.567430    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-hb8td_d8bf6d68-c828-41db-995e-6484b8cbf930/kube-state-metrics/0.log"
Jul 17 05:46:39.595133 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:39.595102    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-hb8td_d8bf6d68-c828-41db-995e-6484b8cbf930/kube-rbac-proxy-main/0.log"
Jul 17 05:46:39.624514 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:39.624469    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-hb8td_d8bf6d68-c828-41db-995e-6484b8cbf930/kube-rbac-proxy-self/0.log"
Jul 17 05:46:39.652122 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:39.652099    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-64f677f8cd-7x5kz_7ff86f69-6040-4003-89d8-84ed6d165084/metrics-server/0.log"
Jul 17 05:46:39.678318 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:39.678231    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-kzwxs_8c609620-1794-4451-8eaa-f9bc3440ea0b/monitoring-plugin/0.log"
Jul 17 05:46:39.786864 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:39.786830    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jtngp_a16c8d78-1778-44b1-acb1-f8e609e3b331/node-exporter/0.log"
Jul 17 05:46:39.805800 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:39.805775    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jtngp_a16c8d78-1778-44b1-acb1-f8e609e3b331/kube-rbac-proxy/0.log"
Jul 17 05:46:39.824605 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:39.824580    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jtngp_a16c8d78-1778-44b1-acb1-f8e609e3b331/init-textfile/0.log"
Jul 17 05:46:39.919331 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:39.919299    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-8td74_07582083-1a89-49cd-904e-09a6df8d62fe/kube-rbac-proxy-main/0.log"
Jul 17 05:46:39.943319 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:39.943295    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-8td74_07582083-1a89-49cd-904e-09a6df8d62fe/kube-rbac-proxy-self/0.log"
Jul 17 05:46:39.965726 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:39.965693    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-8td74_07582083-1a89-49cd-904e-09a6df8d62fe/openshift-state-metrics/0.log"
Jul 17 05:46:40.008772 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:40.008745    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b6151930-7f52-4dad-8aa4-553bcad70b77/prometheus/0.log"
Jul 17 05:46:40.031223 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:40.031192    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b6151930-7f52-4dad-8aa4-553bcad70b77/config-reloader/0.log"
Jul 17 05:46:40.052398 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:40.052364    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b6151930-7f52-4dad-8aa4-553bcad70b77/thanos-sidecar/0.log"
Jul 17 05:46:40.074872 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:40.074846    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b6151930-7f52-4dad-8aa4-553bcad70b77/kube-rbac-proxy-web/0.log"
Jul 17 05:46:40.098500 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:40.098465    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b6151930-7f52-4dad-8aa4-553bcad70b77/kube-rbac-proxy/0.log"
Jul 17 05:46:40.120370 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:40.120343    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b6151930-7f52-4dad-8aa4-553bcad70b77/kube-rbac-proxy-thanos/0.log"
Jul 17 05:46:40.140789 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:40.140764    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b6151930-7f52-4dad-8aa4-553bcad70b77/init-config-reloader/0.log"
Jul 17 05:46:40.177156 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:40.177120    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-nqj6x_49508546-6acb-4970-a9ae-25fad03768fd/prometheus-operator/0.log"
Jul 17 05:46:40.195568 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:40.195477    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-nqj6x_49508546-6acb-4970-a9ae-25fad03768fd/kube-rbac-proxy/0.log"
Jul 17 05:46:40.219778 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:40.219751    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-kxf2j_87aa6ee8-0708-428a-985b-40dbd41f57cb/prometheus-operator-admission-webhook/0.log"
Jul 17 05:46:40.343787 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:40.343758    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7875fbb676-p8ln5_679b67c5-405f-4561-9846-8aaef06ee7bc/thanos-query/0.log"
Jul 17 05:46:40.365461 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:40.365433    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7875fbb676-p8ln5_679b67c5-405f-4561-9846-8aaef06ee7bc/kube-rbac-proxy-web/0.log"
Jul 17 05:46:40.387130 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:40.387098    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7875fbb676-p8ln5_679b67c5-405f-4561-9846-8aaef06ee7bc/kube-rbac-proxy/0.log"
Jul 17 05:46:40.407202 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:40.407173    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7875fbb676-p8ln5_679b67c5-405f-4561-9846-8aaef06ee7bc/prom-label-proxy/0.log"
Jul 17 05:46:40.428884 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:40.428857    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7875fbb676-p8ln5_679b67c5-405f-4561-9846-8aaef06ee7bc/kube-rbac-proxy-rules/0.log"
Jul 17 05:46:40.449801 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:40.449731    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7875fbb676-p8ln5_679b67c5-405f-4561-9846-8aaef06ee7bc/kube-rbac-proxy-metrics/0.log"
Jul 17 05:46:42.525144 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:42.525111    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fvgtg/perf-node-gather-daemonset-7b6lz"]
Jul 17 05:46:42.525793 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:42.525773    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="806ece69-9272-4c82-bb09-cd9f33d884c3" containerName="revoke-keys"
Jul 17 05:46:42.525908 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:42.525892    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="806ece69-9272-4c82-bb09-cd9f33d884c3" containerName="revoke-keys"
Jul 17 05:46:42.525997 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:42.525917    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="806ece69-9272-4c82-bb09-cd9f33d884c3" containerName="revoke-keys"
Jul 17 05:46:42.525997 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:42.525931    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="806ece69-9272-4c82-bb09-cd9f33d884c3" containerName="revoke-keys"
Jul 17 05:46:42.525997 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:42.525982    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="806ece69-9272-4c82-bb09-cd9f33d884c3" containerName="revoke-keys"
Jul 17 05:46:42.525997 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:42.525991    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="806ece69-9272-4c82-bb09-cd9f33d884c3" containerName="revoke-keys"
Jul 17 05:46:42.526200 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:42.526105    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="806ece69-9272-4c82-bb09-cd9f33d884c3" containerName="revoke-keys"
Jul 17 05:46:42.526200 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:42.526123    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="806ece69-9272-4c82-bb09-cd9f33d884c3" containerName="revoke-keys"
Jul 17 05:46:42.530089 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:42.530069    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fvgtg/perf-node-gather-daemonset-7b6lz"
Jul 17 05:46:42.532889 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:42.532870    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fvgtg\"/\"kube-root-ca.crt\""
Jul 17 05:46:42.533017 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:42.532885    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fvgtg\"/\"openshift-service-ca.crt\""
Jul 17 05:46:42.533017 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:42.532868    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-fvgtg\"/\"default-dockercfg-78785\""
Jul 17 05:46:42.536225 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:42.536197    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fvgtg/perf-node-gather-daemonset-7b6lz"]
Jul 17 05:46:42.632751 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:42.632699    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8e4ed611-24e5-4ba2-b66f-9955fdde98ee-sys\") pod \"perf-node-gather-daemonset-7b6lz\" (UID: \"8e4ed611-24e5-4ba2-b66f-9955fdde98ee\") " pod="openshift-must-gather-fvgtg/perf-node-gather-daemonset-7b6lz"
Jul 17 05:46:42.632947 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:42.632771    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8e4ed611-24e5-4ba2-b66f-9955fdde98ee-proc\") pod \"perf-node-gather-daemonset-7b6lz\" (UID: \"8e4ed611-24e5-4ba2-b66f-9955fdde98ee\") " pod="openshift-must-gather-fvgtg/perf-node-gather-daemonset-7b6lz"
Jul 17 05:46:42.632947 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:42.632876    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8e4ed611-24e5-4ba2-b66f-9955fdde98ee-podres\") pod \"perf-node-gather-daemonset-7b6lz\" (UID: \"8e4ed611-24e5-4ba2-b66f-9955fdde98ee\") " pod="openshift-must-gather-fvgtg/perf-node-gather-daemonset-7b6lz"
Jul 17 05:46:42.632947 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:42.632907    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8e4ed611-24e5-4ba2-b66f-9955fdde98ee-lib-modules\") pod \"perf-node-gather-daemonset-7b6lz\" (UID: \"8e4ed611-24e5-4ba2-b66f-9955fdde98ee\") " pod="openshift-must-gather-fvgtg/perf-node-gather-daemonset-7b6lz"
Jul 17 05:46:42.632947 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:42.632940    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2dmd\" (UniqueName: \"kubernetes.io/projected/8e4ed611-24e5-4ba2-b66f-9955fdde98ee-kube-api-access-q2dmd\") pod \"perf-node-gather-daemonset-7b6lz\" (UID: \"8e4ed611-24e5-4ba2-b66f-9955fdde98ee\") " pod="openshift-must-gather-fvgtg/perf-node-gather-daemonset-7b6lz"
Jul 17 05:46:42.733699 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:42.733646    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8e4ed611-24e5-4ba2-b66f-9955fdde98ee-lib-modules\") pod \"perf-node-gather-daemonset-7b6lz\" (UID: \"8e4ed611-24e5-4ba2-b66f-9955fdde98ee\") " pod="openshift-must-gather-fvgtg/perf-node-gather-daemonset-7b6lz"
Jul 17 05:46:42.733699 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:42.733707    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2dmd\" (UniqueName: \"kubernetes.io/projected/8e4ed611-24e5-4ba2-b66f-9955fdde98ee-kube-api-access-q2dmd\") pod \"perf-node-gather-daemonset-7b6lz\" (UID: \"8e4ed611-24e5-4ba2-b66f-9955fdde98ee\") " pod="openshift-must-gather-fvgtg/perf-node-gather-daemonset-7b6lz"
Jul 17 05:46:42.733934 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:42.733762    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8e4ed611-24e5-4ba2-b66f-9955fdde98ee-sys\") pod \"perf-node-gather-daemonset-7b6lz\" (UID: \"8e4ed611-24e5-4ba2-b66f-9955fdde98ee\") " pod="openshift-must-gather-fvgtg/perf-node-gather-daemonset-7b6lz"
Jul 17 05:46:42.733934 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:42.733808    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8e4ed611-24e5-4ba2-b66f-9955fdde98ee-proc\") pod \"perf-node-gather-daemonset-7b6lz\" (UID: \"8e4ed611-24e5-4ba2-b66f-9955fdde98ee\") " pod="openshift-must-gather-fvgtg/perf-node-gather-daemonset-7b6lz"
Jul 17 05:46:42.733934 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:42.733838    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8e4ed611-24e5-4ba2-b66f-9955fdde98ee-lib-modules\") pod \"perf-node-gather-daemonset-7b6lz\" (UID: \"8e4ed611-24e5-4ba2-b66f-9955fdde98ee\") " pod="openshift-must-gather-fvgtg/perf-node-gather-daemonset-7b6lz"
Jul 17 05:46:42.733934 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:42.733868    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8e4ed611-24e5-4ba2-b66f-9955fdde98ee-sys\") pod \"perf-node-gather-daemonset-7b6lz\" (UID: \"8e4ed611-24e5-4ba2-b66f-9955fdde98ee\") " pod="openshift-must-gather-fvgtg/perf-node-gather-daemonset-7b6lz"
Jul 17 05:46:42.733934 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:42.733875    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8e4ed611-24e5-4ba2-b66f-9955fdde98ee-proc\") pod \"perf-node-gather-daemonset-7b6lz\" (UID: \"8e4ed611-24e5-4ba2-b66f-9955fdde98ee\") " pod="openshift-must-gather-fvgtg/perf-node-gather-daemonset-7b6lz"
Jul 17 05:46:42.733934 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:42.733923    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8e4ed611-24e5-4ba2-b66f-9955fdde98ee-podres\") pod \"perf-node-gather-daemonset-7b6lz\" (UID: \"8e4ed611-24e5-4ba2-b66f-9955fdde98ee\") " pod="openshift-must-gather-fvgtg/perf-node-gather-daemonset-7b6lz"
Jul 17 05:46:42.734176 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:42.734039    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8e4ed611-24e5-4ba2-b66f-9955fdde98ee-podres\") pod \"perf-node-gather-daemonset-7b6lz\" (UID: \"8e4ed611-24e5-4ba2-b66f-9955fdde98ee\") " pod="openshift-must-gather-fvgtg/perf-node-gather-daemonset-7b6lz"
Jul 17 05:46:42.741720 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:42.741691    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2dmd\" (UniqueName: \"kubernetes.io/projected/8e4ed611-24e5-4ba2-b66f-9955fdde98ee-kube-api-access-q2dmd\") pod \"perf-node-gather-daemonset-7b6lz\" (UID: \"8e4ed611-24e5-4ba2-b66f-9955fdde98ee\") " pod="openshift-must-gather-fvgtg/perf-node-gather-daemonset-7b6lz"
Jul 17 05:46:42.840990 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:42.840930    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fvgtg/perf-node-gather-daemonset-7b6lz"
Jul 17 05:46:42.981617 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:42.981575    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fvgtg/perf-node-gather-daemonset-7b6lz"]
Jul 17 05:46:42.983776 ip-10-0-132-19 kubenswrapper[2562]: W0717 05:46:42.983746    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8e4ed611_24e5_4ba2_b66f_9955fdde98ee.slice/crio-43c698a12633ffa3e02e190355a9fd9ad1df71b2d62200184cf3de1eb2f6a117 WatchSource:0}: Error finding container 43c698a12633ffa3e02e190355a9fd9ad1df71b2d62200184cf3de1eb2f6a117: Status 404 returned error can't find the container with id 43c698a12633ffa3e02e190355a9fd9ad1df71b2d62200184cf3de1eb2f6a117
Jul 17 05:46:43.301450 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:43.301360    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fvgtg/perf-node-gather-daemonset-7b6lz" event={"ID":"8e4ed611-24e5-4ba2-b66f-9955fdde98ee","Type":"ContainerStarted","Data":"01ed598faae4e73022cc526382c05a2bce43cb774a07ab82ee6de123ee8c16d1"}
Jul 17 05:46:43.301450 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:43.301422    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fvgtg/perf-node-gather-daemonset-7b6lz" event={"ID":"8e4ed611-24e5-4ba2-b66f-9955fdde98ee","Type":"ContainerStarted","Data":"43c698a12633ffa3e02e190355a9fd9ad1df71b2d62200184cf3de1eb2f6a117"}
Jul 17 05:46:43.301640 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:43.301526    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-fvgtg/perf-node-gather-daemonset-7b6lz"
Jul 17 05:46:43.317484 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:43.317435    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-fvgtg/perf-node-gather-daemonset-7b6lz" podStartSLOduration=1.317418332 podStartE2EDuration="1.317418332s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 05:46:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:46:43.315669332 +0000 UTC m=+3390.999263018" watchObservedRunningTime="2026-07-17 05:46:43.317418332 +0000 UTC m=+3391.001011989"
Jul 17 05:46:44.600429 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:44.600398    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-kf2nj_9b4a2877-135f-4be6-a249-83c2006db349/dns/0.log"
Jul 17 05:46:44.618437 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:44.618410    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-kf2nj_9b4a2877-135f-4be6-a249-83c2006db349/kube-rbac-proxy/0.log"
Jul 17 05:46:44.708977 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:44.708946    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4l4dn_052c1550-58cb-4558-8b40-d9bd51a597b9/dns-node-resolver/0.log"
Jul 17 05:46:45.300847 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:45.300815    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bwdpn_43b05be8-ccbf-412d-ad62-74596e1710fa/node-ca/0.log"
Jul 17 05:46:45.878389 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:45.878355    2562 scope.go:117] "RemoveContainer" containerID="8d531b7ce2a5a319ed5b059865bc3c192f727bb125a4e24a96c76e5e3d73f49b"
Jul 17 05:46:46.321571 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:46.321524    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-r6hhr" event={"ID":"e228b24d-8a8d-4c35-a2bd-52392292acbf","Type":"ContainerStarted","Data":"c3c88fafa300b18233fed1a042255644912895c70195027f4ce4a1282d56cf5f"}
Jul 17 05:46:46.341224 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:46.341166    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-r6hhr" podStartSLOduration=17.341145057 podStartE2EDuration="17.341145057s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 05:46:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 05:46:46.337888489 +0000 UTC m=+3394.021482170" watchObservedRunningTime="2026-07-17 05:46:46.341145057 +0000 UTC m=+3394.024738717"
Jul 17 05:46:46.468046 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:46.467937    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-9tb57_e0a7089b-2cde-439a-8cd8-187338f02f18/discovery/0.log"
Jul 17 05:46:46.545715 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:46.545678    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-k2snx_162f43aa-9c5c-4148-8f41-4c1f43c9d7a9/istio-proxy/0.log"
Jul 17 05:46:46.607743 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:46.607712    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-k2snx_162f43aa-9c5c-4148-8f41-4c1f43c9d7a9/istio-proxy/1.log"
Jul 17 05:46:46.644417 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:46.644383    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-qc8sv_fc5f1ecb-28e8-49ce-89d7-b841f10e04b4/payload-pre-processing/0.log"
Jul 17 05:46:47.327981 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:47.327922    2562 generic.go:356] "Generic (PLEG): container finished" podID="e228b24d-8a8d-4c35-a2bd-52392292acbf" containerID="c3c88fafa300b18233fed1a042255644912895c70195027f4ce4a1282d56cf5f" exitCode=6
Jul 17 05:46:47.328409 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:47.328001    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-r6hhr" event={"ID":"e228b24d-8a8d-4c35-a2bd-52392292acbf","Type":"ContainerDied","Data":"c3c88fafa300b18233fed1a042255644912895c70195027f4ce4a1282d56cf5f"}
Jul 17 05:46:47.328409 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:47.328062    2562 scope.go:117] "RemoveContainer" containerID="8d531b7ce2a5a319ed5b059865bc3c192f727bb125a4e24a96c76e5e3d73f49b"
Jul 17 05:46:47.328524 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:47.328413    2562 scope.go:117] "RemoveContainer" containerID="c3c88fafa300b18233fed1a042255644912895c70195027f4ce4a1282d56cf5f"
Jul 17 05:46:47.328693 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:46:47.328665    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-c8a383ce-r6hhr_odh-ai-gateway-infra(e228b24d-8a8d-4c35-a2bd-52392292acbf)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-r6hhr" podUID="e228b24d-8a8d-4c35-a2bd-52392292acbf"
Jul 17 05:46:47.352077 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:47.352048    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-r6hhr"]
Jul 17 05:46:47.369471 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:47.369436    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8pbjc_3f3f79dc-2048-413f-a701-1ac6a22554a4/serve-healthcheck-canary/0.log"
Jul 17 05:46:47.886038 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:47.885942    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pwfdn_5ddbfecb-13e8-4c64-942b-6b8100d6b92a/kube-rbac-proxy/0.log"
Jul 17 05:46:47.904737 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:47.904711    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pwfdn_5ddbfecb-13e8-4c64-942b-6b8100d6b92a/exporter/0.log"
Jul 17 05:46:47.924766 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:47.924740    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pwfdn_5ddbfecb-13e8-4c64-942b-6b8100d6b92a/extractor/0.log"
Jul 17 05:46:48.471373 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:48.471347    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-r6hhr"
Jul 17 05:46:48.587848 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:48.587747    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e228b24d-8a8d-4c35-a2bd-52392292acbf-maas-api-service-ca\") pod \"e228b24d-8a8d-4c35-a2bd-52392292acbf\" (UID: \"e228b24d-8a8d-4c35-a2bd-52392292acbf\") "
Jul 17 05:46:48.588223 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:48.588191    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e228b24d-8a8d-4c35-a2bd-52392292acbf-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e228b24d-8a8d-4c35-a2bd-52392292acbf" (UID: "e228b24d-8a8d-4c35-a2bd-52392292acbf"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:46:48.688705 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:48.688675    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e228b24d-8a8d-4c35-a2bd-52392292acbf-maas-api-service-ca\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:46:49.316885 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:49.316807    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-fvgtg/perf-node-gather-daemonset-7b6lz"
Jul 17 05:46:49.341355 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:49.341326    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-r6hhr" event={"ID":"e228b24d-8a8d-4c35-a2bd-52392292acbf","Type":"ContainerDied","Data":"6034ce92c5a2ec0260cf054c984fa3abd4520603fadb96db9e29935211efccdd"}
Jul 17 05:46:49.341532 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:49.341380    2562 scope.go:117] "RemoveContainer" containerID="c3c88fafa300b18233fed1a042255644912895c70195027f4ce4a1282d56cf5f"
Jul 17 05:46:49.341532 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:49.341334    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-r6hhr"
Jul 17 05:46:49.357929 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:49.357906    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-r6hhr"]
Jul 17 05:46:49.364995 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:49.364955    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c8a383ce-r6hhr"]
Jul 17 05:46:50.452633 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:50.452596    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-859ff64978-dq8x9_8f3b63a1-b0a2-42d4-ab11-11ad32132d44/manager/0.log"
Jul 17 05:46:50.610505 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:50.610471    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d857c78d5-ft2q8_a667f213-da2d-4290-b67c-700500da24b0/manager/0.log"
Jul 17 05:46:50.628163 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:50.628142    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d857c78d5-ft2q8_a667f213-da2d-4290-b67c-700500da24b0/copy-manifests/0.log"
Jul 17 05:46:50.878396 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:50.878367    2562 scope.go:117] "RemoveContainer" containerID="ba09c9c6da293e81b851b632a1b82fd620c4d2e08de2f1da9e196724cbb72687"
Jul 17 05:46:50.883951 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:50.883921    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e228b24d-8a8d-4c35-a2bd-52392292acbf" path="/var/lib/kubelet/pods/e228b24d-8a8d-4c35-a2bd-52392292acbf/volumes"
Jul 17 05:46:51.354457 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:51.354422    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-8x5rn" event={"ID":"2c2bb0e6-2065-457e-96e8-27a6170fba95","Type":"ContainerStarted","Data":"90142b94743e2435484d4b909d0aac035e0a8cfaf859da4758c9e8c3362da72e"}
Jul 17 05:46:52.117374 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:52.117350    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-559bdd8b4c-gj9kt_a242308f-aa18-435e-a83c-3eb68deff925/manager/0.log"
Jul 17 05:46:52.361803 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:52.361767    2562 generic.go:356] "Generic (PLEG): container finished" podID="2c2bb0e6-2065-457e-96e8-27a6170fba95" containerID="90142b94743e2435484d4b909d0aac035e0a8cfaf859da4758c9e8c3362da72e" exitCode=6
Jul 17 05:46:52.362004 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:52.361841    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-8x5rn" event={"ID":"2c2bb0e6-2065-457e-96e8-27a6170fba95","Type":"ContainerDied","Data":"90142b94743e2435484d4b909d0aac035e0a8cfaf859da4758c9e8c3362da72e"}
Jul 17 05:46:52.362004 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:52.361905    2562 scope.go:117] "RemoveContainer" containerID="ba09c9c6da293e81b851b632a1b82fd620c4d2e08de2f1da9e196724cbb72687"
Jul 17 05:46:52.362223 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:52.362199    2562 scope.go:117] "RemoveContainer" containerID="90142b94743e2435484d4b909d0aac035e0a8cfaf859da4758c9e8c3362da72e"
Jul 17 05:46:52.362467 ip-10-0-132-19 kubenswrapper[2562]: E0717 05:46:52.362445    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-bf0ad9db-8x5rn_odh-ai-gateway-infra(2c2bb0e6-2065-457e-96e8-27a6170fba95)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-8x5rn" podUID="2c2bb0e6-2065-457e-96e8-27a6170fba95"
Jul 17 05:46:52.383431 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:52.383403    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-8x5rn"]
Jul 17 05:46:53.266123 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:53.266087    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-9tb57_e0a7089b-2cde-439a-8cd8-187338f02f18/discovery/0.log"
Jul 17 05:46:53.323882 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:53.323845    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-k2snx_162f43aa-9c5c-4148-8f41-4c1f43c9d7a9/istio-proxy/0.log"
Jul 17 05:46:53.394644 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:53.394616    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-k2snx_162f43aa-9c5c-4148-8f41-4c1f43c9d7a9/istio-proxy/1.log"
Jul 17 05:46:53.428696 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:53.428663    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-qc8sv_fc5f1ecb-28e8-49ce-89d7-b841f10e04b4/payload-pre-processing/0.log"
Jul 17 05:46:53.532977 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:53.532938    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-8x5rn"
Jul 17 05:46:53.632892 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:53.632857    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c2bb0e6-2065-457e-96e8-27a6170fba95-maas-api-service-ca\") pod \"2c2bb0e6-2065-457e-96e8-27a6170fba95\" (UID: \"2c2bb0e6-2065-457e-96e8-27a6170fba95\") "
Jul 17 05:46:53.633260 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:53.633235    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c2bb0e6-2065-457e-96e8-27a6170fba95-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2c2bb0e6-2065-457e-96e8-27a6170fba95" (UID: "2c2bb0e6-2065-457e-96e8-27a6170fba95"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 05:46:53.734399 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:53.734366    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c2bb0e6-2065-457e-96e8-27a6170fba95-maas-api-service-ca\") on node \"ip-10-0-132-19.ec2.internal\" DevicePath \"\""
Jul 17 05:46:54.377327 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:54.377289    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-8x5rn"
Jul 17 05:46:54.377774 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:54.377348    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-8x5rn" event={"ID":"2c2bb0e6-2065-457e-96e8-27a6170fba95","Type":"ContainerDied","Data":"d72ee8f44a4ab97c2c726462b0ef28197d932c9d3ac12ae8163f9b4f56599bbb"}
Jul 17 05:46:54.377774 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:54.377418    2562 scope.go:117] "RemoveContainer" containerID="90142b94743e2435484d4b909d0aac035e0a8cfaf859da4758c9e8c3362da72e"
Jul 17 05:46:54.399137 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:54.399108    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-8x5rn"]
Jul 17 05:46:54.404660 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:54.404638    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bf0ad9db-8x5rn"]
Jul 17 05:46:54.876901 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:54.876864    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-9tb57_e0a7089b-2cde-439a-8cd8-187338f02f18/discovery/0.log"
Jul 17 05:46:54.883696 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:54.883661    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c2bb0e6-2065-457e-96e8-27a6170fba95" path="/var/lib/kubelet/pods/2c2bb0e6-2065-457e-96e8-27a6170fba95/volumes"
Jul 17 05:46:54.930892 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:54.930862    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-k2snx_162f43aa-9c5c-4148-8f41-4c1f43c9d7a9/istio-proxy/0.log"
Jul 17 05:46:54.994445 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:54.994411    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-k2snx_162f43aa-9c5c-4148-8f41-4c1f43c9d7a9/istio-proxy/1.log"
Jul 17 05:46:55.020359 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:55.020329    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-qc8sv_fc5f1ecb-28e8-49ce-89d7-b841f10e04b4/payload-pre-processing/0.log"
Jul 17 05:46:58.500310 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:58.500285    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-644hn_01097a42-523b-4764-b6a6-bf531915f32f/kube-multus-additional-cni-plugins/0.log"
Jul 17 05:46:58.523571 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:58.523544    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-644hn_01097a42-523b-4764-b6a6-bf531915f32f/egress-router-binary-copy/0.log"
Jul 17 05:46:58.543280 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:58.543251    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-644hn_01097a42-523b-4764-b6a6-bf531915f32f/cni-plugins/0.log"
Jul 17 05:46:58.561938 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:58.561911    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-644hn_01097a42-523b-4764-b6a6-bf531915f32f/bond-cni-plugin/0.log"
Jul 17 05:46:58.581627 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:58.581599    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-644hn_01097a42-523b-4764-b6a6-bf531915f32f/routeoverride-cni/0.log"
Jul 17 05:46:58.601459 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:58.601438    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-644hn_01097a42-523b-4764-b6a6-bf531915f32f/whereabouts-cni-bincopy/0.log"
Jul 17 05:46:58.620718 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:58.620693    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-644hn_01097a42-523b-4764-b6a6-bf531915f32f/whereabouts-cni/0.log"
Jul 17 05:46:58.993768 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:58.993721    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-lbjqq_f4c22119-9902-4e1f-bc76-97d51253b16c/kube-multus/0.log"
Jul 17 05:46:59.185652 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:59.185607    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-rzdct_a54b09be-6889-45d6-a748-68e5b94b73d2/network-metrics-daemon/0.log"
Jul 17 05:46:59.202279 ip-10-0-132-19 kubenswrapper[2562]: I0717 05:46:59.202248    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-rzdct_a54b09be-6889-45d6-a748-68e5b94b73d2/kube-rbac-proxy/0.log"