Jul 23 23:13:30.496033 ip-10-0-129-165 systemd[1]: Starting Kubernetes Kubelet...
Jul 23 23:13:30.932668 ip-10-0-129-165 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 23 23:13:30.932668 ip-10-0-129-165 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 23 23:13:30.932668 ip-10-0-129-165 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 23 23:13:30.932668 ip-10-0-129-165 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 23 23:13:30.932668 ip-10-0-129-165 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 23 23:13:30.935398 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.935265    2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 23 23:13:30.938918 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.938900    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 23:13:30.938918 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.938919    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 23:13:30.938987 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.938923    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 23:13:30.938987 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.938926    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 23:13:30.938987 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.938929    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 23:13:30.938987 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.938932    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 23:13:30.938987 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.938935    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 23:13:30.938987 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.938938    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 23:13:30.938987 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.938941    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 23:13:30.938987 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.938943    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 23:13:30.938987 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.938946    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 23:13:30.938987 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.938948    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 23:13:30.938987 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.938951    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 23:13:30.938987 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.938954    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 23:13:30.938987 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.938957    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 23:13:30.938987 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.938960    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 23:13:30.938987 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.938963    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 23:13:30.938987 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.938965    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 23:13:30.938987 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.938968    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 23:13:30.938987 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.938970    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 23:13:30.938987 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.938973    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 23:13:30.938987 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.938975    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 23:13:30.939473 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.938978    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 23:13:30.939473 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.938980    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 23:13:30.939473 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.938983    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 23:13:30.939473 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.938985    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 23:13:30.939473 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.938988    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 23:13:30.939473 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.938991    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 23:13:30.939473 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.938994    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 23:13:30.939473 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.938997    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 23:13:30.939473 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939000    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 23:13:30.939473 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939003    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 23:13:30.939473 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939005    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 23:13:30.939473 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939008    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 23:13:30.939473 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939010    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 23:13:30.939473 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939013    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 23:13:30.939473 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939015    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 23:13:30.939473 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939018    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 23:13:30.939473 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939021    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 23:13:30.939473 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939023    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 23:13:30.939473 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939031    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 23:13:30.939473 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939034    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 23:13:30.939986 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939036    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 23:13:30.939986 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939039    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 23:13:30.939986 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939041    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 23:13:30.939986 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939043    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 23:13:30.939986 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939048    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 23:13:30.939986 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939053    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 23:13:30.939986 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939056    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 23:13:30.939986 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939058    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 23:13:30.939986 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939061    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 23:13:30.939986 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939063    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 23:13:30.939986 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939065    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 23:13:30.939986 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939068    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 23:13:30.939986 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939070    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 23:13:30.939986 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939075    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 23:13:30.939986 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939079    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 23:13:30.939986 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939082    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 23:13:30.939986 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939085    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 23:13:30.939986 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939087    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 23:13:30.939986 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939090    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 23:13:30.940447 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939092    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 23:13:30.940447 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939095    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 23:13:30.940447 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939097    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 23:13:30.940447 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939100    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 23:13:30.940447 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939102    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 23:13:30.940447 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939105    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 23:13:30.940447 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939108    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 23:13:30.940447 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939110    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 23:13:30.940447 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939112    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 23:13:30.940447 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939115    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 23:13:30.940447 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939118    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 23:13:30.940447 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939120    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 23:13:30.940447 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939123    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 23:13:30.940447 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939125    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 23:13:30.940447 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939128    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 23:13:30.940447 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939130    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 23:13:30.940447 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939132    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 23:13:30.940447 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939135    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 23:13:30.940447 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939137    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 23:13:30.940447 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939139    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 23:13:30.940996 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939142    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 23:13:30.940996 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939144    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 23:13:30.940996 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939146    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 23:13:30.940996 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939149    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 23:13:30.940996 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939151    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 23:13:30.940996 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939154    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 23:13:30.940996 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939160    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 23:13:30.940996 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939163    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 23:13:30.940996 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939166    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 23:13:30.940996 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939169    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 23:13:30.940996 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939172    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 23:13:30.940996 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939175    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 23:13:30.940996 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939194    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 23:13:30.940996 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939197    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 23:13:30.940996 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939200    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 23:13:30.940996 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939203    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 23:13:30.940996 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939205    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 23:13:30.940996 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939208    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 23:13:30.940996 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939935    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 23:13:30.941457 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939944    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 23:13:30.941457 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939948    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 23:13:30.941457 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939951    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 23:13:30.941457 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939954    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 23:13:30.941457 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939957    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 23:13:30.941457 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939960    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 23:13:30.941457 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939963    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 23:13:30.941457 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939966    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 23:13:30.941457 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939968    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 23:13:30.941457 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939971    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 23:13:30.941457 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939974    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 23:13:30.941457 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939976    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 23:13:30.941457 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939986    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 23:13:30.941457 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939989    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 23:13:30.941457 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939992    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 23:13:30.941457 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939995    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 23:13:30.941457 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.939997    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 23:13:30.941457 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940000    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 23:13:30.941457 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940003    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 23:13:30.941940 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940006    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 23:13:30.941940 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940009    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 23:13:30.941940 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940012    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 23:13:30.941940 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940014    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 23:13:30.941940 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940017    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 23:13:30.941940 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940020    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 23:13:30.941940 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940022    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 23:13:30.941940 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940025    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 23:13:30.941940 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940027    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 23:13:30.941940 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940030    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 23:13:30.941940 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940032    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 23:13:30.941940 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940035    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 23:13:30.941940 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940038    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 23:13:30.941940 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940040    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 23:13:30.941940 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940043    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 23:13:30.941940 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940045    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 23:13:30.941940 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940048    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 23:13:30.941940 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940050    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 23:13:30.941940 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940054    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 23:13:30.941940 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940057    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 23:13:30.942422 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940059    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 23:13:30.942422 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940064    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 23:13:30.942422 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940067    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 23:13:30.942422 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940070    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 23:13:30.942422 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940072    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 23:13:30.942422 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940075    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 23:13:30.942422 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940078    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 23:13:30.942422 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940081    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 23:13:30.942422 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940084    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 23:13:30.942422 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940086    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 23:13:30.942422 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940089    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 23:13:30.942422 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940092    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 23:13:30.942422 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940095    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 23:13:30.942422 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940098    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 23:13:30.942422 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940100    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 23:13:30.942422 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940103    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 23:13:30.942422 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940105    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 23:13:30.942422 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940108    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 23:13:30.942422 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940110    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 23:13:30.942422 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940113    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 23:13:30.942913 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940115    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 23:13:30.942913 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940118    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 23:13:30.942913 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940120    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 23:13:30.942913 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940123    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 23:13:30.942913 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940125    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 23:13:30.942913 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940128    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 23:13:30.942913 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940130    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 23:13:30.942913 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940133    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 23:13:30.942913 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940135    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 23:13:30.942913 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940137    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 23:13:30.942913 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940140    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 23:13:30.942913 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940143    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 23:13:30.942913 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940145    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 23:13:30.942913 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940147    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 23:13:30.942913 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940150    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 23:13:30.942913 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940152    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 23:13:30.942913 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940154    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 23:13:30.942913 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940157    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 23:13:30.942913 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940159    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 23:13:30.942913 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940162    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 23:13:30.943492 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940164    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 23:13:30.943492 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940166    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 23:13:30.943492 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940169    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 23:13:30.943492 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940171    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 23:13:30.943492 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940175    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 23:13:30.943492 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940178    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 23:13:30.943492 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940182    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 23:13:30.943492 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940184    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 23:13:30.943492 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940187    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 23:13:30.943492 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940190    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 23:13:30.943492 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940192    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 23:13:30.943492 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940194    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 23:13:30.943492 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940198    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 23:13:30.943492 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940202    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 23:13:30.943492 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940204    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 23:13:30.943492 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940207    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 23:13:30.943492 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940209    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 23:13:30.943492 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940211    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 23:13:30.943492 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940214    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 23:13:30.943492 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940599    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 23:13:30.943992 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940603    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 23:13:30.943992 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940606    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 23:13:30.943992 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940608    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 23:13:30.943992 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940611    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 23:13:30.943992 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940614    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 23:13:30.943992 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940617    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 23:13:30.943992 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940619    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 23:13:30.943992 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940622    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 23:13:30.943992 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940624    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 23:13:30.943992 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940627    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 23:13:30.943992 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940629    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 23:13:30.943992 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940632    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 23:13:30.943992 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940634    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 23:13:30.943992 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940637    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 23:13:30.943992 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940639    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 23:13:30.943992 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940642    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 23:13:30.943992 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940645    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 23:13:30.943992 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940664    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 23:13:30.943992 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940667    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 23:13:30.943992 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940670    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 23:13:30.944525 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940672    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 23:13:30.944525 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940675    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 23:13:30.944525 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940677    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 23:13:30.944525 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940680    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 23:13:30.944525 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940683    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 23:13:30.944525 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940686    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 23:13:30.944525 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940688    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 23:13:30.944525 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940691    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 23:13:30.944525 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940693    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 23:13:30.944525 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940696    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 23:13:30.944525 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940699    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 23:13:30.944525 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940703    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 23:13:30.944525 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940706    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 23:13:30.944525 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940708    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 23:13:30.944525 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940711    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 23:13:30.944525 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940714    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 23:13:30.944525 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940717    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 23:13:30.944525 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940720    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 23:13:30.944525 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940722    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 23:13:30.944525 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940726    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 23:13:30.945033 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940729    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 23:13:30.945033 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940731    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 23:13:30.945033 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940734    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 23:13:30.945033 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940737    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 23:13:30.945033 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940739    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 23:13:30.945033 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940742    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 23:13:30.945033 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940744    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 23:13:30.945033 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940747    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 23:13:30.945033 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940749    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 23:13:30.945033 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940752    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 23:13:30.945033 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940754    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 23:13:30.945033 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940757    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 23:13:30.945033 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940759    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 23:13:30.945033 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940762    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 23:13:30.945033 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940764    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 23:13:30.945033 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940767    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 23:13:30.945033 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940770    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 23:13:30.945033 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940773    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 23:13:30.945033 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940775    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 23:13:30.945033 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940777    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 23:13:30.945514 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940780    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 23:13:30.945514 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940782    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 23:13:30.945514 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940785    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 23:13:30.945514 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940787    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 23:13:30.945514 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940789    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 23:13:30.945514 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940792    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 23:13:30.945514 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940794    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 23:13:30.945514 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940797    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 23:13:30.945514 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940800    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 23:13:30.945514 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940803    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 23:13:30.945514 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940806    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 23:13:30.945514 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940808    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 23:13:30.945514 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940813    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 23:13:30.945514 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940817    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 23:13:30.945514 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940820    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 23:13:30.945514 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940823    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 23:13:30.945514 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940826    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 23:13:30.945514 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940829    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 23:13:30.945514 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940831    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 23:13:30.946027 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940834    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 23:13:30.946027 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940836    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 23:13:30.946027 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940839    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 23:13:30.946027 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940841    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 23:13:30.946027 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940845    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 23:13:30.946027 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940848    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 23:13:30.946027 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940850    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 23:13:30.946027 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940853    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 23:13:30.946027 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940857    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 23:13:30.946027 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940860    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 23:13:30.946027 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940862    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 23:13:30.946027 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940865    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 23:13:30.946027 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940868    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 23:13:30.946027 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940870    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 23:13:30.946027 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940873    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 23:13:30.946027 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940875    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 23:13:30.946027 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940878    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 23:13:30.946027 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940881    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 23:13:30.946027 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.940883    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 23:13:30.946478 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.940964    2578 flags.go:64] FLAG: --address="0.0.0.0"
Jul 23 23:13:30.946478 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.940979    2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 23 23:13:30.946478 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.940986    2578 flags.go:64] FLAG: --anonymous-auth="true"
Jul 23 23:13:30.946478 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.940990    2578 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 23 23:13:30.946478 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.940995    2578 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 23 23:13:30.946478 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.940998    2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 23 23:13:30.946478 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941003    2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 23 23:13:30.946478 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941007    2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 23 23:13:30.946478 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941011    2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 23 23:13:30.946478 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941014    2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 23 23:13:30.946478 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941017    2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 23 23:13:30.946478 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941020    2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 23 23:13:30.946478 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941023    2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 23 23:13:30.946478 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941026    2578 flags.go:64] FLAG: --cgroup-root=""
Jul 23 23:13:30.946478 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941029    2578 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 23 23:13:30.946478 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941032    2578 flags.go:64] FLAG: --client-ca-file=""
Jul 23 23:13:30.946478 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941034    2578 flags.go:64] FLAG: --cloud-provider="external"
Jul 23 23:13:30.946478 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941037    2578 flags.go:64] FLAG: --cluster-dns="[]"
Jul 23 23:13:30.946478 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941042    2578 flags.go:64] FLAG: --cluster-domain=""
Jul 23 23:13:30.946478 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941044    2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 23 23:13:30.946478 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941047    2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 23 23:13:30.946478 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941053    2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 23 23:13:30.946478 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941056    2578 flags.go:64] FLAG: --container-log-max-files="5"
Jul 23 23:13:30.946478 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941060    2578 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 23 23:13:30.947074 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941063    2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 23 23:13:30.947074 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941066    2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 23 23:13:30.947074 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941069    2578 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 23 23:13:30.947074 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941072    2578 flags.go:64] FLAG: --contention-profiling="false"
Jul 23 23:13:30.947074 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941075    2578 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 23 23:13:30.947074 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941079    2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 23 23:13:30.947074 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941082    2578 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 23 23:13:30.947074 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941085    2578 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 23 23:13:30.947074 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941089    2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 23 23:13:30.947074 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941092    2578 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 23 23:13:30.947074 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941095    2578 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 23 23:13:30.947074 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941097    2578 flags.go:64] FLAG: --enable-load-reader="false"
Jul 23 23:13:30.947074 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941100    2578 flags.go:64] FLAG: --enable-server="true"
Jul 23 23:13:30.947074 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941105    2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 23 23:13:30.947074 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941109    2578 flags.go:64] FLAG: --event-burst="100"
Jul 23 23:13:30.947074 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941112    2578 flags.go:64] FLAG: --event-qps="50"
Jul 23 23:13:30.947074 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941115    2578 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 23 23:13:30.947074 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941117    2578 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 23 23:13:30.947074 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941120    2578 flags.go:64] FLAG: --eviction-hard=""
Jul 23 23:13:30.947074 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941124    2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 23 23:13:30.947074 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941127    2578 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 23 23:13:30.947074 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941130    2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 23 23:13:30.947074 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941133    2578 flags.go:64] FLAG: --eviction-soft=""
Jul 23 23:13:30.947074 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941135    2578 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 23 23:13:30.947074 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941138    2578 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 23 23:13:30.947734 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941141    2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 23 23:13:30.947734 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941144    2578 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 23 23:13:30.947734 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941147    2578 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 23 23:13:30.947734 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941149    2578 flags.go:64] FLAG: --fail-swap-on="true"
Jul 23 23:13:30.947734 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941153    2578 flags.go:64] FLAG: --feature-gates=""
Jul 23 23:13:30.947734 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941157    2578 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 23 23:13:30.947734 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941160    2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 23 23:13:30.947734 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941163    2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 23 23:13:30.947734 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941166    2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 23 23:13:30.947734 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941169    2578 flags.go:64] FLAG: --healthz-port="10248"
Jul 23 23:13:30.947734 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941172    2578 flags.go:64] FLAG: --help="false"
Jul 23 23:13:30.947734 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941175    2578 flags.go:64] FLAG: --hostname-override="ip-10-0-129-165.ec2.internal"
Jul 23 23:13:30.947734 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941179    2578 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 23 23:13:30.947734 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941182    2578 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 23 23:13:30.947734 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941185    2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 23 23:13:30.947734 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941189    2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 23 23:13:30.947734 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941192    2578 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 23 23:13:30.947734 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941195    2578 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 23 23:13:30.947734 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941197    2578 flags.go:64] FLAG: --image-service-endpoint=""
Jul 23 23:13:30.947734 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941201    2578 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 23 23:13:30.947734 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941203    2578 flags.go:64] FLAG: --kube-api-burst="100"
Jul 23 23:13:30.947734 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941206    2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 23 23:13:30.947734 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941210    2578 flags.go:64] FLAG: --kube-api-qps="50"
Jul 23 23:13:30.947734 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941213    2578 flags.go:64] FLAG: --kube-reserved=""
Jul 23 23:13:30.948339 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941215    2578 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 23 23:13:30.948339 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941218    2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 23 23:13:30.948339 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941221    2578 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 23 23:13:30.948339 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941224    2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 23 23:13:30.948339 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941226    2578 flags.go:64] FLAG: --lock-file=""
Jul 23 23:13:30.948339 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941229    2578 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 23 23:13:30.948339 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941232    2578 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 23 23:13:30.948339 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941234    2578 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 23 23:13:30.948339 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941239    2578 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 23 23:13:30.948339 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941242    2578 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 23 23:13:30.948339 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941245    2578 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 23 23:13:30.948339 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941248    2578 flags.go:64] FLAG: --logging-format="text"
Jul 23 23:13:30.948339 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941251    2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 23 23:13:30.948339 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941255    2578 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 23 23:13:30.948339 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941258    2578 flags.go:64] FLAG: --manifest-url=""
Jul 23 23:13:30.948339 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941261    2578 flags.go:64] FLAG: --manifest-url-header=""
Jul 23 23:13:30.948339 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941265    2578 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 23 23:13:30.948339 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941268    2578 flags.go:64] FLAG: --max-open-files="1000000"
Jul 23 23:13:30.948339 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941272    2578 flags.go:64] FLAG: --max-pods="110"
Jul 23 23:13:30.948339 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941275    2578 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 23 23:13:30.948339 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941280    2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 23 23:13:30.948339 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941283    2578 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 23 23:13:30.948339 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941286    2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 23 23:13:30.948339 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941289    2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 23 23:13:30.948339 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941292    2578 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 23 23:13:30.949289 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941295    2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 23 23:13:30.949289 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941303    2578 flags.go:64] FLAG: --node-status-max-images="50"
Jul 23 23:13:30.949289 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941306    2578 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 23 23:13:30.949289 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941309    2578 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 23 23:13:30.949289 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941312    2578 flags.go:64] FLAG: --pod-cidr=""
Jul 23 23:13:30.949289 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941317    2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 23 23:13:30.949289 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941322    2578 flags.go:64] FLAG: --pod-manifest-path=""
Jul 23 23:13:30.949289 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941325    2578 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 23 23:13:30.949289 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941328    2578 flags.go:64] FLAG: --pods-per-core="0"
Jul 23 23:13:30.949289 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941331    2578 flags.go:64] FLAG: --port="10250"
Jul 23 23:13:30.949289 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941334    2578 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 23 23:13:30.949289 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941337    2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-05eb33607c43517ba"
Jul 23 23:13:30.949289 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941340    2578 flags.go:64] FLAG: --qos-reserved=""
Jul 23 23:13:30.949289 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941343    2578 flags.go:64] FLAG: --read-only-port="10255"
Jul 23 23:13:30.949289 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941346    2578 flags.go:64] FLAG: --register-node="true"
Jul 23 23:13:30.949289 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941349    2578 flags.go:64] FLAG: --register-with-taints=""
Jul 23 23:13:30.949289 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941353    2578 flags.go:64] FLAG: --registry-burst="10"
Jul 23 23:13:30.949289 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941356    2578 flags.go:64] FLAG: --registry-qps="5"
Jul 23 23:13:30.949289 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941358    2578 flags.go:64] FLAG: --reserved-cpus=""
Jul 23 23:13:30.949289 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941362    2578 flags.go:64] FLAG: --reserved-memory=""
Jul 23 23:13:30.949289 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941366    2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 23 23:13:30.949289 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941369    2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 23 23:13:30.949289 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941372    2578 flags.go:64] FLAG: --rotate-certificates="false"
Jul 23 23:13:30.949289 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941375    2578 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 23 23:13:30.949289 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941378    2578 flags.go:64] FLAG: --runonce="false"
Jul 23 23:13:30.949955 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941380    2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 23 23:13:30.949955 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941383    2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 23 23:13:30.949955 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941386    2578 flags.go:64] FLAG: --seccomp-default="false"
Jul 23 23:13:30.949955 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941390    2578 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 23 23:13:30.949955 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941393    2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 23 23:13:30.949955 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941396    2578 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 23 23:13:30.949955 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941399    2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 23 23:13:30.949955 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941402    2578 flags.go:64] FLAG: --storage-driver-password="root"
Jul 23 23:13:30.949955 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941405    2578 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 23 23:13:30.949955 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941408    2578 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 23 23:13:30.949955 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941410    2578 flags.go:64] FLAG: --storage-driver-user="root"
Jul 23 23:13:30.949955 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941413    2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 23 23:13:30.949955 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941416    2578 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 23 23:13:30.949955 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941420    2578 flags.go:64] FLAG: --system-cgroups=""
Jul 23 23:13:30.949955 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941422    2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 23 23:13:30.949955 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941427    2578 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 23 23:13:30.949955 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941430    2578 flags.go:64] FLAG: --tls-cert-file=""
Jul 23 23:13:30.949955 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941433    2578 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 23 23:13:30.949955 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941437    2578 flags.go:64] FLAG: --tls-min-version=""
Jul 23 23:13:30.949955 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941440    2578 flags.go:64] FLAG: --tls-private-key-file=""
Jul 23 23:13:30.949955 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941442    2578 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 23 23:13:30.949955 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941445    2578 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 23 23:13:30.949955 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941448    2578 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 23 23:13:30.949955 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941451    2578 flags.go:64] FLAG: --v="2"
Jul 23 23:13:30.949955 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941457    2578 flags.go:64] FLAG: --version="false"
Jul 23 23:13:30.950556 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941461    2578 flags.go:64] FLAG: --vmodule=""
Jul 23 23:13:30.950556 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941466    2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 23 23:13:30.950556 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941469    2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 23 23:13:30.950556 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941667    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 23:13:30.950556 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941672    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 23:13:30.950556 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941675    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 23:13:30.950556 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941678    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 23:13:30.950556 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941681    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 23:13:30.950556 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941684    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 23:13:30.950556 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941686    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 23:13:30.950556 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941690    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 23:13:30.950556 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941693    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 23:13:30.950556 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941696    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 23:13:30.950556 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941699    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 23:13:30.950556 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941701    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 23:13:30.950556 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941704    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 23:13:30.950556 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941707    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 23:13:30.950556 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941709    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 23:13:30.950556 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941712    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 23:13:30.950556 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941715    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 23:13:30.951155 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941717    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 23:13:30.951155 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941720    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 23:13:30.951155 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941723    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 23:13:30.951155 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941725    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 23:13:30.951155 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941727    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 23:13:30.951155 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941731    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 23:13:30.951155 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941733    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 23:13:30.951155 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941736    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 23:13:30.951155 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941738    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 23:13:30.951155 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941740    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 23:13:30.951155 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941743    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 23:13:30.951155 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941745    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 23:13:30.951155 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941748    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 23:13:30.951155 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941751    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 23:13:30.951155 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941754    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 23:13:30.951155 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941757    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 23:13:30.951155 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941759    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 23:13:30.951155 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941761    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 23:13:30.951155 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941765    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 23:13:30.951155 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941769    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 23:13:30.951643 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941772    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 23:13:30.951643 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941775    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 23:13:30.951643 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941780    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 23:13:30.951643 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941783    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 23:13:30.951643 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941786    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 23:13:30.951643 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941788    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 23:13:30.951643 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941791    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 23:13:30.951643 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941793    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 23:13:30.951643 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941796    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 23:13:30.951643 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941798    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 23:13:30.951643 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941801    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 23:13:30.951643 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941804    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 23:13:30.951643 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941806    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 23:13:30.951643 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941808    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 23:13:30.951643 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941811    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 23:13:30.951643 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941814    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 23:13:30.951643 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941816    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 23:13:30.951643 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941819    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 23:13:30.951643 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941821    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 23:13:30.952131 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941824    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 23:13:30.952131 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941826    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 23:13:30.952131 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941829    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 23:13:30.952131 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941831    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 23:13:30.952131 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941833    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 23:13:30.952131 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941836    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 23:13:30.952131 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941841    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 23:13:30.952131 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941844    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 23:13:30.952131 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941846    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 23:13:30.952131 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941848    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 23:13:30.952131 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941851    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 23:13:30.952131 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941854    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 23:13:30.952131 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941856    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 23:13:30.952131 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941859    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 23:13:30.952131 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941861    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 23:13:30.952131 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941865    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 23:13:30.952131 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941867    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 23:13:30.952131 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941870    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 23:13:30.952131 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941872    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 23:13:30.952131 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941875    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 23:13:30.952626 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941877    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 23:13:30.952626 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941880    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 23:13:30.952626 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941882    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 23:13:30.952626 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941886    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 23:13:30.952626 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941889    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 23:13:30.952626 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941893    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 23:13:30.952626 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941895    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 23:13:30.952626 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941898    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 23:13:30.952626 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941902    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 23:13:30.952626 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941904    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 23:13:30.952626 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941906    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 23:13:30.952626 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941909    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 23:13:30.952626 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941911    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 23:13:30.952626 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941914    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 23:13:30.952626 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941916    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 23:13:30.952626 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941919    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 23:13:30.952626 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941921    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 23:13:30.952626 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941924    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 23:13:30.952626 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941928    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 23:13:30.952626 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941931    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 23:13:30.953125 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941934    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 23:13:30.953125 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941937    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 23:13:30.953125 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.941939    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 23:13:30.953125 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.941944    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 23 23:13:30.953125 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.950504    2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 23 23:13:30.953125 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.950521    2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 23 23:13:30.953125 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.950542    2578 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 23 23:13:30.953125 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.950550    2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 23 23:13:30.953125 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950610    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 23:13:30.953125 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950615    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 23:13:30.953125 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950619    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 23:13:30.953125 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950622    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 23:13:30.953125 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950625    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 23:13:30.953125 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950628    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 23:13:30.953476 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950631    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 23:13:30.953476 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950634    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 23:13:30.953476 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950638    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 23:13:30.953476 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950640    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 23:13:30.953476 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950643    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 23:13:30.953476 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950646    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 23:13:30.953476 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950667    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 23:13:30.953476 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950673    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 23:13:30.953476 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950675    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 23:13:30.953476 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950678    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 23:13:30.953476 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950680    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 23:13:30.953476 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950683    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 23:13:30.953476 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950685    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 23:13:30.953476 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950688    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 23:13:30.953476 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950691    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 23:13:30.953476 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950694    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 23:13:30.953476 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950697    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 23:13:30.953476 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950700    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 23:13:30.953476 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950703    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 23:13:30.953476 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950705    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 23:13:30.953990 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950708    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 23:13:30.953990 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950712    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 23:13:30.953990 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950715    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 23:13:30.953990 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950718    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 23:13:30.953990 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950720    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 23:13:30.953990 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950723    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 23:13:30.953990 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950728    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 23:13:30.953990 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950731    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 23:13:30.953990 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950734    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 23:13:30.953990 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950737    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 23:13:30.953990 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950739    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 23:13:30.953990 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950742    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 23:13:30.953990 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950744    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 23:13:30.953990 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950747    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 23:13:30.953990 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950750    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 23:13:30.953990 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950752    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 23:13:30.953990 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950755    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 23:13:30.953990 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950757    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 23:13:30.953990 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950760    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 23:13:30.954449 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950762    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 23:13:30.954449 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950765    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 23:13:30.954449 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950767    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 23:13:30.954449 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950770    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 23:13:30.954449 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950772    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 23:13:30.954449 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950775    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 23:13:30.954449 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950777    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 23:13:30.954449 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950779    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 23:13:30.954449 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950782    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 23:13:30.954449 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950785    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 23:13:30.954449 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950787    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 23:13:30.954449 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950790    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 23:13:30.954449 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950792    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 23:13:30.954449 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950794    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 23:13:30.954449 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950797    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 23:13:30.954449 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950800    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 23:13:30.954449 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950802    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 23:13:30.954449 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950804    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 23:13:30.954449 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950807    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 23:13:30.954449 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950810    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 23:13:30.955036 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950812    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 23:13:30.955036 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950815    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 23:13:30.955036 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950817    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 23:13:30.955036 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950820    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 23:13:30.955036 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950822    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 23:13:30.955036 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950825    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 23:13:30.955036 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950827    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 23:13:30.955036 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950830    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 23:13:30.955036 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950832    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 23:13:30.955036 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950835    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 23:13:30.955036 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950837    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 23:13:30.955036 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950840    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 23:13:30.955036 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950842    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 23:13:30.955036 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950844    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 23:13:30.955036 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950847    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 23:13:30.955036 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950850    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 23:13:30.955036 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950853    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 23:13:30.955036 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950855    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 23:13:30.955036 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950858    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 23:13:30.955515 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950860    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 23:13:30.955515 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950863    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 23:13:30.955515 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950865    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 23:13:30.955515 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950868    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 23:13:30.955515 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950870    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 23:13:30.955515 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950873    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 23:13:30.955515 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950875    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 23:13:30.955515 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950878    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 23:13:30.955515 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950880    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 23:13:30.955515 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950883    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 23:13:30.955515 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950885    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 23:13:30.955515 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950887    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 23:13:30.955515 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950892    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 23:13:30.955515 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950895    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 23:13:30.955515 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.950898    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 23:13:30.955916 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.950904    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 23 23:13:30.955916 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951020    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 23:13:30.955916 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951025    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 23:13:30.955916 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951028    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 23:13:30.955916 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951031    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 23:13:30.955916 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951033    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 23:13:30.955916 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951036    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 23:13:30.955916 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951039    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 23:13:30.955916 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951041    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 23:13:30.955916 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951043    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 23:13:30.955916 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951046    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 23:13:30.955916 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951048    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 23:13:30.955916 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951051    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 23:13:30.955916 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951054    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 23:13:30.955916 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951056    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 23:13:30.956295 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951059    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 23:13:30.956295 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951061    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 23:13:30.956295 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951063    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 23:13:30.956295 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951066    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 23:13:30.956295 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951068    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 23:13:30.956295 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951071    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 23:13:30.956295 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951073    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 23:13:30.956295 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951075    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 23:13:30.956295 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951078    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 23:13:30.956295 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951080    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 23:13:30.956295 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951083    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 23:13:30.956295 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951086    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 23:13:30.956295 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951090    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 23:13:30.956295 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951093    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 23:13:30.956295 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951095    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 23:13:30.956295 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951099    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 23:13:30.956295 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951101    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 23:13:30.956295 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951104    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 23:13:30.956295 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951107    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 23:13:30.956777 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951110    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 23:13:30.956777 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951114    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 23:13:30.956777 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951116    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 23:13:30.956777 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951119    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 23:13:30.956777 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951121    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 23:13:30.956777 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951124    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 23:13:30.956777 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951126    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 23:13:30.956777 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951129    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 23:13:30.956777 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951131    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 23:13:30.956777 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951133    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 23:13:30.956777 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951136    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 23:13:30.956777 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951138    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 23:13:30.956777 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951141    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 23:13:30.956777 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951143    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 23:13:30.956777 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951145    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 23:13:30.956777 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951148    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 23:13:30.956777 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951150    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 23:13:30.956777 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951153    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 23:13:30.956777 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951156    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 23:13:30.956777 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951158    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 23:13:30.957255 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951161    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 23:13:30.957255 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951164    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 23:13:30.957255 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951166    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 23:13:30.957255 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951169    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 23:13:30.957255 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951172    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 23:13:30.957255 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951174    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 23:13:30.957255 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951177    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 23:13:30.957255 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951179    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 23:13:30.957255 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951182    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 23:13:30.957255 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951184    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 23:13:30.957255 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951187    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 23:13:30.957255 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951189    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 23:13:30.957255 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951191    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 23:13:30.957255 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951194    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 23:13:30.957255 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951196    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 23:13:30.957255 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951199    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 23:13:30.957255 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951201    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 23:13:30.957255 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951204    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 23:13:30.957255 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951206    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 23:13:30.957255 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951209    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 23:13:30.957772 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951211    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 23:13:30.957772 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951214    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 23:13:30.957772 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951217    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 23:13:30.957772 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951221    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 23:13:30.957772 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951224    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 23:13:30.957772 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951227    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 23:13:30.957772 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951230    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 23:13:30.957772 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951232    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 23:13:30.957772 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951235    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 23:13:30.957772 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951238    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 23:13:30.957772 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951241    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 23:13:30.957772 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951243    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 23:13:30.957772 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951246    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 23:13:30.957772 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951249    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 23:13:30.957772 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951251    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 23:13:30.957772 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951254    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 23:13:30.957772 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951256    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 23:13:30.957772 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951259    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 23:13:30.957772 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951261    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 23:13:30.958223 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951264    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 23:13:30.958223 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951266    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 23:13:30.958223 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951268    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 23:13:30.958223 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951271    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 23:13:30.958223 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951273    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 23:13:30.958223 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951276    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 23:13:30.958223 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:30.951278    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 23:13:30.958223 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.951283    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 23 23:13:30.958223 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.951432    2578 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 23 23:13:30.958223 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.955646    2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 23 23:13:30.958223 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.957047    2578 server.go:1019] "Starting client certificate rotation"
Jul 23 23:13:30.958223 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.957160    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 23 23:13:30.958223 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.957205    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 23 23:13:30.960186 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.960171    2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 23 23:13:30.963995 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.963968    2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 23 23:13:30.973901 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.973875    2578 log.go:25] "Validated CRI v1 runtime API"
Jul 23 23:13:30.976242 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.976222    2578 log.go:25] "Validated CRI v1 image API"
Jul 23 23:13:30.977102 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.977088    2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 23 23:13:30.980579 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.980554    2578 fs.go:135] Filesystem UUIDs: map[21b48788-5d2e-4abe-9a30-8bfef4a2fb81:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 fe21e521-20d8-499f-8234-5d51ddc44951:/dev/nvme0n1p4]
Jul 23 23:13:30.980628 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.980579    2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 23 23:13:30.987417 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.987278    2578 manager.go:216] Machine: {Timestamp:2026-07-23 23:13:30.984979294 +0000 UTC m=+0.354504944 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3090074 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b18fdceb93f5b7f30c2d193c25253 SystemUUID:ec2b18fd-ceb9-3f5b-7f30-c2d193c25253 BootID:5ad7bc8e-9161-46b5-bd7b-4f601aac7239 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:5c:92:1f:06:f1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:5c:92:1f:06:f1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:3a:2c:05:99:65:c8 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 23 23:13:30.987417 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.987411    2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 23 23:13:30.987530 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.987514    2578 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 23 23:13:30.990686 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.990667    2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 23 23:13:30.991946 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.991908    2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 23 23:13:30.992135 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.991950    2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-165.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 23 23:13:30.993395 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.993379    2578 topology_manager.go:138] "Creating topology manager with none policy"
Jul 23 23:13:30.993450 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.993404    2578 container_manager_linux.go:309] "Creating device plugin manager"
Jul 23 23:13:30.993450 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.993422    2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 23 23:13:30.994202 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.994189    2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 23 23:13:30.995009 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.994996    2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 23 23:13:30.996926 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.996910    2578 state_mem.go:36] "Initialized new in-memory state store"
Jul 23 23:13:30.997145 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.997134    2578 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 23 23:13:30.997186 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.997169    2578 kubelet.go:486] "Attempting to sync node with API server"
Jul 23 23:13:30.997217 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.997198    2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 23 23:13:30.997248 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.997219    2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 23 23:13:30.997248 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.997245    2578 kubelet.go:398] "Adding apiserver pod source"
Jul 23 23:13:30.997306 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.997258    2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 23 23:13:30.997816 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.997793    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 23 23:13:30.997816 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:30.997819    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 23 23:13:31.001191 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.001149    2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 23 23:13:31.002998 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.002979    2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 23 23:13:31.003106 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.003010    2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 23 23:13:31.004701 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.004687    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 23 23:13:31.004769 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.004704    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 23 23:13:31.004769 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.004710    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 23 23:13:31.004769 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.004716    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 23 23:13:31.004769 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.004731    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 23 23:13:31.004769 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.004738    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 23 23:13:31.004769 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.004746    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 23 23:13:31.004769 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.004751    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 23 23:13:31.004769 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.004758    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 23 23:13:31.004769 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.004763    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 23 23:13:31.004769 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.004772    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 23 23:13:31.005032 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.004778    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 23 23:13:31.006627 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.006608    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 23 23:13:31.006627 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.006623    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 23 23:13:31.009528 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.009509    2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-165.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 23 23:13:31.009784 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:31.009761    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 23 23:13:31.009858 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:31.009834    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-165.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 23 23:13:31.010121 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.010109    2578 server.go:1268] "Started kubelet"
Jul 23 23:13:31.010278 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.010216    2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 23 23:13:31.010365 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.010316    2578 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 23 23:13:31.010418 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.010390    2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 23 23:13:31.011039 ip-10-0-129-165 systemd[1]: Started Kubernetes Kubelet.
Jul 23 23:13:31.012321 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.012301    2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 23 23:13:31.013108 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.013088    2578 server.go:310] "Adding debug handlers to kubelet server"
Jul 23 23:13:31.013443 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.013427    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-j49dx"
Jul 23 23:13:31.015954 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:31.014974    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-165.ec2.internal.18c50e2c5c2d0e09  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-165.ec2.internal,UID:ip-10-0-129-165.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-165.ec2.internal,},FirstTimestamp:2026-07-23 23:13:31.010076169 +0000 UTC m=+0.379601817,LastTimestamp:2026-07-23 23:13:31.010076169 +0000 UTC m=+0.379601817,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-165.ec2.internal,}"
Jul 23 23:13:31.016642 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:31.016623    2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 23 23:13:31.019477 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.019459    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-j49dx"
Jul 23 23:13:31.020179 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.020164    2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 23 23:13:31.020263 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.020165    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 23 23:13:31.020382 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:31.020334    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-165.ec2.internal\" not found"
Jul 23 23:13:31.020778 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.020763    2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 23 23:13:31.021036 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.021018    2578 factory.go:55] Registering systemd factory
Jul 23 23:13:31.021113 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.021044    2578 factory.go:223] Registration of the systemd container factory successfully
Jul 23 23:13:31.021426 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.021246    2578 factory.go:153] Registering CRI-O factory
Jul 23 23:13:31.021426 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.021260    2578 factory.go:223] Registration of the crio container factory successfully
Jul 23 23:13:31.021426 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.021274    2578 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 23 23:13:31.021426 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.021287    2578 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 23 23:13:31.021426 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.021336    2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 23 23:13:31.021426 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.021410    2578 reconstruct.go:97] "Volume reconstruction finished"
Jul 23 23:13:31.021426 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.021425    2578 reconciler.go:29] "Reconciler: start to sync state"
Jul 23 23:13:31.021731 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.021457    2578 factory.go:103] Registering Raw factory
Jul 23 23:13:31.021731 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.021482    2578 manager.go:1196] Started watching for new ooms in manager
Jul 23 23:13:31.022082 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.022074    2578 manager.go:349] Starting recovery of all containers
Jul 23 23:13:31.031946 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.031916    2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 23:13:31.033245 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.033219    2578 manager.go:354] Recovery completed
Jul 23 23:13:31.035431 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:31.035241    2578 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-129-165.ec2.internal\" not found" node="ip-10-0-129-165.ec2.internal"
Jul 23 23:13:31.039265 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.039248    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 23:13:31.041066 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.041049    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-165.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 23:13:31.041148 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.041084    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-165.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 23:13:31.041148 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.041095    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-165.ec2.internal" event="NodeHasSufficientPID"
Jul 23 23:13:31.041685 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.041666    2578 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 23 23:13:31.041685 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.041684    2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 23 23:13:31.041789 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.041700    2578 state_mem.go:36] "Initialized new in-memory state store"
Jul 23 23:13:31.041789 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.041675    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 23:13:31.042549 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.042533    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-165.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 23:13:31.042595 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.042563    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-165.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 23:13:31.042595 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.042574    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-165.ec2.internal" event="NodeHasSufficientPID"
Jul 23 23:13:31.043492 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.043480    2578 policy_none.go:49] "None policy: Start"
Jul 23 23:13:31.043532 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.043497    2578 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 23 23:13:31.043532 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.043508    2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 23 23:13:31.045365 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.045353    2578 policy_none.go:47] "Start"
Jul 23 23:13:31.079399 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.079375    2578 manager.go:337] "Starting Device Plugin manager"
Jul 23 23:13:31.079533 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:31.079417    2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 23 23:13:31.079533 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.079428    2578 server.go:87] "Starting device plugin registration server"
Jul 23 23:13:31.079786 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.079773    2578 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 23 23:13:31.079843 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.079787    2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 23 23:13:31.079894 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.079874    2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 23 23:13:31.080009 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.079991    2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 23 23:13:31.080009 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.080006    2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 23 23:13:31.080698 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:31.080670    2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 23 23:13:31.080769 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:31.080727    2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-165.ec2.internal\" not found"
Jul 23 23:13:31.180926 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.180879    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 23:13:31.182158 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.181999    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-165.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 23:13:31.182158 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.182033    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-165.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 23:13:31.182158 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.182049    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-165.ec2.internal" event="NodeHasSufficientPID"
Jul 23 23:13:31.182158 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.182079    2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-165.ec2.internal"
Jul 23 23:13:31.190547 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.190498    2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-165.ec2.internal"
Jul 23 23:13:31.190547 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:31.190524    2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-165.ec2.internal\": node \"ip-10-0-129-165.ec2.internal\" not found"
Jul 23 23:13:31.215948 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:31.215914    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-165.ec2.internal\" not found"
Jul 23 23:13:31.228635 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.228586    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 23 23:13:31.229961 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.229935    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 23 23:13:31.230079 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.229976    2578 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 23 23:13:31.230079 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.230001    2578 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 23 23:13:31.230079 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:31.230040    2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 23 23:13:31.232519 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.232494    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 23:13:31.316748 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:31.316704    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-165.ec2.internal\" not found"
Jul 23 23:13:31.331169 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.331119    2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-165.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-165.ec2.internal"]
Jul 23 23:13:31.331255 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.331216    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 23:13:31.332602 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.332584    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-165.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 23:13:31.332695 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.332614    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-165.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 23:13:31.332695 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.332630    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-165.ec2.internal" event="NodeHasSufficientPID"
Jul 23 23:13:31.334135 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.334123    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 23:13:31.334259 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.334245    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-165.ec2.internal"
Jul 23 23:13:31.334309 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.334273    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 23:13:31.334912 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.334894    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-165.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 23:13:31.335001 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.334927    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-165.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 23:13:31.335001 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.334940    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-165.ec2.internal" event="NodeHasSufficientPID"
Jul 23 23:13:31.335001 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.334943    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-165.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 23:13:31.335001 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.334966    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-165.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 23:13:31.335001 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.334979    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-165.ec2.internal" event="NodeHasSufficientPID"
Jul 23 23:13:31.336222 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.336205    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-165.ec2.internal"
Jul 23 23:13:31.336276 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.336239    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 23:13:31.336933 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.336919    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-165.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 23:13:31.337020 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.336949    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-165.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 23:13:31.337020 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.336963    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-165.ec2.internal" event="NodeHasSufficientPID"
Jul 23 23:13:31.363611 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:31.363583    2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-165.ec2.internal\" not found" node="ip-10-0-129-165.ec2.internal"
Jul 23 23:13:31.368241 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:31.368221    2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-165.ec2.internal\" not found" node="ip-10-0-129-165.ec2.internal"
Jul 23 23:13:31.416852 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:31.416811    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-165.ec2.internal\" not found"
Jul 23 23:13:31.423209 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.423186    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ef9bd6a40b72bafbfb43200d1ac0a01d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-165.ec2.internal\" (UID: \"ef9bd6a40b72bafbfb43200d1ac0a01d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-165.ec2.internal"
Jul 23 23:13:31.423306 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.423218    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ef9bd6a40b72bafbfb43200d1ac0a01d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-165.ec2.internal\" (UID: \"ef9bd6a40b72bafbfb43200d1ac0a01d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-165.ec2.internal"
Jul 23 23:13:31.423306 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.423255    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f2a7eb7430412c2da89feeebacf09842-config\") pod \"kube-apiserver-proxy-ip-10-0-129-165.ec2.internal\" (UID: \"f2a7eb7430412c2da89feeebacf09842\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-165.ec2.internal"
Jul 23 23:13:31.517085 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:31.517016    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-165.ec2.internal\" not found"
Jul 23 23:13:31.524383 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.524356    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f2a7eb7430412c2da89feeebacf09842-config\") pod \"kube-apiserver-proxy-ip-10-0-129-165.ec2.internal\" (UID: \"f2a7eb7430412c2da89feeebacf09842\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-165.ec2.internal"
Jul 23 23:13:31.524444 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.524387    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ef9bd6a40b72bafbfb43200d1ac0a01d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-165.ec2.internal\" (UID: \"ef9bd6a40b72bafbfb43200d1ac0a01d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-165.ec2.internal"
Jul 23 23:13:31.524444 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.524403    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ef9bd6a40b72bafbfb43200d1ac0a01d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-165.ec2.internal\" (UID: \"ef9bd6a40b72bafbfb43200d1ac0a01d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-165.ec2.internal"
Jul 23 23:13:31.524539 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.524446    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ef9bd6a40b72bafbfb43200d1ac0a01d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-165.ec2.internal\" (UID: \"ef9bd6a40b72bafbfb43200d1ac0a01d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-165.ec2.internal"
Jul 23 23:13:31.524539 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.524462    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f2a7eb7430412c2da89feeebacf09842-config\") pod \"kube-apiserver-proxy-ip-10-0-129-165.ec2.internal\" (UID: \"f2a7eb7430412c2da89feeebacf09842\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-165.ec2.internal"
Jul 23 23:13:31.524539 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.524477    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ef9bd6a40b72bafbfb43200d1ac0a01d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-165.ec2.internal\" (UID: \"ef9bd6a40b72bafbfb43200d1ac0a01d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-165.ec2.internal"
Jul 23 23:13:31.617572 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:31.617531    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-165.ec2.internal\" not found"
Jul 23 23:13:31.666086 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.666056    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-165.ec2.internal"
Jul 23 23:13:31.672147 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.672126    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-165.ec2.internal"
Jul 23 23:13:31.718306 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:31.718267    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-165.ec2.internal\" not found"
Jul 23 23:13:31.818827 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:31.818765    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-165.ec2.internal\" not found"
Jul 23 23:13:31.916743 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.916711    2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 23:13:31.920380 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.920360    2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-165.ec2.internal"
Jul 23 23:13:31.933393 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.933374    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 23 23:13:31.935369 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.935355    2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-165.ec2.internal"
Jul 23 23:13:31.943888 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.943868    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 23 23:13:31.957043 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.957010    2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 23 23:13:31.957180 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.957160    2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 23 23:13:31.957219 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.957199    2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 23 23:13:31.957254 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.957200    2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 23 23:13:31.998388 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:31.998358    2578 apiserver.go:52] "Watching apiserver"
Jul 23 23:13:32.015016 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.014980    2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 23 23:13:32.015339 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.015319    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-29zxk","openshift-image-registry/node-ca-ljq8l","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-165.ec2.internal","openshift-multus/multus-f66wp","openshift-multus/network-metrics-daemon-bbv68","kube-system/konnectivity-agent-9jn86","openshift-dns/node-resolver-sn5k2","openshift-multus/multus-additional-cni-plugins-rw58q","openshift-network-diagnostics/network-check-target-5d9sj","openshift-network-operator/iptables-alerter-v6zh5","openshift-ovn-kubernetes/ovnkube-node-mx6j5","kube-system/global-pull-secret-syncer-z8d42","kube-system/kube-apiserver-proxy-ip-10-0-129-165.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzjsz"]
Jul 23 23:13:32.017619 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.017588    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-29zxk"
Jul 23 23:13:32.019015 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.018997    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ljq8l"
Jul 23 23:13:32.020132 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.020111    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 23 23:13:32.020232 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.020219    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 23 23:13:32.020353 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.020337    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 23 23:13:32.020673 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.020647    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-d6wdj\""
Jul 23 23:13:32.021423 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.021386    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.022112 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.022089    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 23 23:13:32.022229 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.022165    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 23 23:13:32.022984 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.022769    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-t5cvp\""
Jul 23 23:13:32.022984 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.022975    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 23 23:13:32.023113 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.023035    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bbv68"
Jul 23 23:13:32.023167 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:32.023127    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bbv68" podUID="8dd112f5-f0cc-4e72-917c-33e4abca79cb"
Jul 23 23:13:32.024407 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.024389    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 23 23:13:32.024903 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.024625    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 23 23:13:32.024903 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.024639    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9jn86"
Jul 23 23:13:32.024903 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.024604    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-22 23:08:31 +0000 UTC" deadline="2028-03-25 00:58:03.754359064 +0000 UTC"
Jul 23 23:13:32.024903 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.024689    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14641h44m31.729674285s"
Jul 23 23:13:32.024903 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.024674    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 23 23:13:32.024903 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.024746    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 23 23:13:32.024903 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.024748    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-2b9vc\""
Jul 23 23:13:32.026465 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.026449    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d1632b98-84f8-4658-af28-6521dd300a59-lib-modules\") pod \"tuned-29zxk\" (UID: \"d1632b98-84f8-4658-af28-6521dd300a59\") " pod="openshift-cluster-node-tuning-operator/tuned-29zxk"
Jul 23 23:13:32.026559 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.026470    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d2943613-62a6-476e-bec2-12fb3c167e01-host\") pod \"node-ca-ljq8l\" (UID: \"d2943613-62a6-476e-bec2-12fb3c167e01\") " pod="openshift-image-registry/node-ca-ljq8l"
Jul 23 23:13:32.026559 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.026485    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d2943613-62a6-476e-bec2-12fb3c167e01-serviceca\") pod \"node-ca-ljq8l\" (UID: \"d2943613-62a6-476e-bec2-12fb3c167e01\") " pod="openshift-image-registry/node-ca-ljq8l"
Jul 23 23:13:32.026559 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.026501    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nsjt\" (UniqueName: \"kubernetes.io/projected/d2943613-62a6-476e-bec2-12fb3c167e01-kube-api-access-7nsjt\") pod \"node-ca-ljq8l\" (UID: \"d2943613-62a6-476e-bec2-12fb3c167e01\") " pod="openshift-image-registry/node-ca-ljq8l"
Jul 23 23:13:32.026559 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.026517    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/59b943fa-86eb-453f-8e2f-63c64ce435ce-host-run-multus-certs\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.026559 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.026532    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d1632b98-84f8-4658-af28-6521dd300a59-etc-sysconfig\") pod \"tuned-29zxk\" (UID: \"d1632b98-84f8-4658-af28-6521dd300a59\") " pod="openshift-cluster-node-tuning-operator/tuned-29zxk"
Jul 23 23:13:32.026816 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.026567    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d1632b98-84f8-4658-af28-6521dd300a59-etc-systemd\") pod \"tuned-29zxk\" (UID: \"d1632b98-84f8-4658-af28-6521dd300a59\") " pod="openshift-cluster-node-tuning-operator/tuned-29zxk"
Jul 23 23:13:32.026816 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.026590    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d1632b98-84f8-4658-af28-6521dd300a59-run\") pod \"tuned-29zxk\" (UID: \"d1632b98-84f8-4658-af28-6521dd300a59\") " pod="openshift-cluster-node-tuning-operator/tuned-29zxk"
Jul 23 23:13:32.026816 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.026605    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/59b943fa-86eb-453f-8e2f-63c64ce435ce-hostroot\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.026816 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.026626    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/59b943fa-86eb-453f-8e2f-63c64ce435ce-cni-binary-copy\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.026816 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.026670    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/59b943fa-86eb-453f-8e2f-63c64ce435ce-multus-socket-dir-parent\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.026816 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.026689    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d1632b98-84f8-4658-af28-6521dd300a59-etc-modprobe-d\") pod \"tuned-29zxk\" (UID: \"d1632b98-84f8-4658-af28-6521dd300a59\") " pod="openshift-cluster-node-tuning-operator/tuned-29zxk"
Jul 23 23:13:32.026816 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.026710    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d1632b98-84f8-4658-af28-6521dd300a59-etc-kubernetes\") pod \"tuned-29zxk\" (UID: \"d1632b98-84f8-4658-af28-6521dd300a59\") " pod="openshift-cluster-node-tuning-operator/tuned-29zxk"
Jul 23 23:13:32.026816 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.026727    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lrrw\" (UniqueName: \"kubernetes.io/projected/d1632b98-84f8-4658-af28-6521dd300a59-kube-api-access-5lrrw\") pod \"tuned-29zxk\" (UID: \"d1632b98-84f8-4658-af28-6521dd300a59\") " pod="openshift-cluster-node-tuning-operator/tuned-29zxk"
Jul 23 23:13:32.026816 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.026767    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/59b943fa-86eb-453f-8e2f-63c64ce435ce-system-cni-dir\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.026816 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.026792    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/59b943fa-86eb-453f-8e2f-63c64ce435ce-os-release\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.027268 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.026835    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/59b943fa-86eb-453f-8e2f-63c64ce435ce-host-var-lib-kubelet\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.027268 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.026864    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d1632b98-84f8-4658-af28-6521dd300a59-etc-sysctl-d\") pod \"tuned-29zxk\" (UID: \"d1632b98-84f8-4658-af28-6521dd300a59\") " pod="openshift-cluster-node-tuning-operator/tuned-29zxk"
Jul 23 23:13:32.027268 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.026888    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d1632b98-84f8-4658-af28-6521dd300a59-sys\") pod \"tuned-29zxk\" (UID: \"d1632b98-84f8-4658-af28-6521dd300a59\") " pod="openshift-cluster-node-tuning-operator/tuned-29zxk"
Jul 23 23:13:32.027268 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.026909    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d1632b98-84f8-4658-af28-6521dd300a59-etc-tuned\") pod \"tuned-29zxk\" (UID: \"d1632b98-84f8-4658-af28-6521dd300a59\") " pod="openshift-cluster-node-tuning-operator/tuned-29zxk"
Jul 23 23:13:32.027268 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.026935    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/59b943fa-86eb-453f-8e2f-63c64ce435ce-multus-cni-dir\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.027268 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.026950    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/59b943fa-86eb-453f-8e2f-63c64ce435ce-host-run-netns\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.027268 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.026964    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/59b943fa-86eb-453f-8e2f-63c64ce435ce-etc-kubernetes\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.027268 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.026977    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d1632b98-84f8-4658-af28-6521dd300a59-var-lib-kubelet\") pod \"tuned-29zxk\" (UID: \"d1632b98-84f8-4658-af28-6521dd300a59\") " pod="openshift-cluster-node-tuning-operator/tuned-29zxk"
Jul 23 23:13:32.027268 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.026997    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d1632b98-84f8-4658-af28-6521dd300a59-tmp\") pod \"tuned-29zxk\" (UID: \"d1632b98-84f8-4658-af28-6521dd300a59\") " pod="openshift-cluster-node-tuning-operator/tuned-29zxk"
Jul 23 23:13:32.027268 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.027021    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/59b943fa-86eb-453f-8e2f-63c64ce435ce-host-var-lib-cni-bin\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.027268 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.027043    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/59b943fa-86eb-453f-8e2f-63c64ce435ce-multus-conf-dir\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.027268 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.027070    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/59b943fa-86eb-453f-8e2f-63c64ce435ce-host-run-k8s-cni-cncf-io\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.027268 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.027087    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 23 23:13:32.027268 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.027093    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/59b943fa-86eb-453f-8e2f-63c64ce435ce-multus-daemon-config\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.027268 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.027119    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swbjc\" (UniqueName: \"kubernetes.io/projected/59b943fa-86eb-453f-8e2f-63c64ce435ce-kube-api-access-swbjc\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.027268 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.027150    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d1632b98-84f8-4658-af28-6521dd300a59-etc-sysctl-conf\") pod \"tuned-29zxk\" (UID: \"d1632b98-84f8-4658-af28-6521dd300a59\") " pod="openshift-cluster-node-tuning-operator/tuned-29zxk"
Jul 23 23:13:32.027268 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.027176    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d1632b98-84f8-4658-af28-6521dd300a59-host\") pod \"tuned-29zxk\" (UID: \"d1632b98-84f8-4658-af28-6521dd300a59\") " pod="openshift-cluster-node-tuning-operator/tuned-29zxk"
Jul 23 23:13:32.027268 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.027190    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/59b943fa-86eb-453f-8e2f-63c64ce435ce-cnibin\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.028005 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.027208    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/59b943fa-86eb-453f-8e2f-63c64ce435ce-host-var-lib-cni-multus\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.028005 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.027479    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sn5k2"
Jul 23 23:13:32.028972 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.028956    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rw58q"
Jul 23 23:13:32.030881 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.030860    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-ttt7t\""
Jul 23 23:13:32.030881 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.030877    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 23 23:13:32.031627 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.031127    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 23 23:13:32.031627 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.031428    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-dbqpq\""
Jul 23 23:13:32.031627 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.031427    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 23 23:13:32.031860 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.031710    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 23 23:13:32.031935 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.031920    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 23 23:13:32.032106 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.032092    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-5g97f\""
Jul 23 23:13:32.032909 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.032891    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5d9sj"
Jul 23 23:13:32.032983 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:32.032961    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5d9sj" podUID="f009ab3e-edd9-4bf3-877e-37115c04d13f"
Jul 23 23:13:32.033672 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.033639    2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 23 23:13:32.038547 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.038528    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-v6zh5"
Jul 23 23:13:32.041391 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.041366    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 23 23:13:32.041391 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.041387    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 23 23:13:32.041565 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.041406    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 23 23:13:32.041565 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.041406    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-zbvlq\""
Jul 23 23:13:32.042322 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.042303    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.044822 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.044804    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 23 23:13:32.044822 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.044829    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 23 23:13:32.045108 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.045091    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 23 23:13:32.045329 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.045311    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-7799t\""
Jul 23 23:13:32.045476 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.045457    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z8d42"
Jul 23 23:13:32.045562 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:32.045536    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z8d42" podUID="b4d53a59-83bb-4725-841c-9f5b55f85ca4"
Jul 23 23:13:32.046504 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.046480    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 23 23:13:32.046504 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.046500    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 23 23:13:32.046655 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.046500    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 23 23:13:32.048355 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.048337    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzjsz"
Jul 23 23:13:32.050990 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.050966    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 23 23:13:32.051083 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.051044    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 23 23:13:32.051083 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.051059    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-kgfkl\""
Jul 23 23:13:32.051171 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.050978    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 23 23:13:32.065363 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.065338    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-h5lns"
Jul 23 23:13:32.074765 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.074654    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-h5lns"
Jul 23 23:13:32.122086 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.122059    2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 23 23:13:32.127399 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.127376    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d1632b98-84f8-4658-af28-6521dd300a59-var-lib-kubelet\") pod \"tuned-29zxk\" (UID: \"d1632b98-84f8-4658-af28-6521dd300a59\") " pod="openshift-cluster-node-tuning-operator/tuned-29zxk"
Jul 23 23:13:32.127522 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.127408    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d1632b98-84f8-4658-af28-6521dd300a59-tmp\") pod \"tuned-29zxk\" (UID: \"d1632b98-84f8-4658-af28-6521dd300a59\") " pod="openshift-cluster-node-tuning-operator/tuned-29zxk"
Jul 23 23:13:32.127522 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.127431    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/59b943fa-86eb-453f-8e2f-63c64ce435ce-host-var-lib-cni-bin\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.127522 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.127460    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c30a49c-1943-4650-83f0-adaa3045404e-tmp-dir\") pod \"node-resolver-sn5k2\" (UID: \"6c30a49c-1943-4650-83f0-adaa3045404e\") " pod="openshift-dns/node-resolver-sn5k2"
Jul 23 23:13:32.127522 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.127485    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-ovn-node-metrics-cert\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.127522 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.127508    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d1632b98-84f8-4658-af28-6521dd300a59-var-lib-kubelet\") pod \"tuned-29zxk\" (UID: \"d1632b98-84f8-4658-af28-6521dd300a59\") " pod="openshift-cluster-node-tuning-operator/tuned-29zxk"
Jul 23 23:13:32.127824 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.127511    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvq8j\" (UniqueName: \"kubernetes.io/projected/61f6356e-0fdc-40c3-be10-216d774ff70b-kube-api-access-lvq8j\") pod \"aws-ebs-csi-driver-node-nzjsz\" (UID: \"61f6356e-0fdc-40c3-be10-216d774ff70b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzjsz"
Jul 23 23:13:32.127824 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.127556    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/59b943fa-86eb-453f-8e2f-63c64ce435ce-host-run-k8s-cni-cncf-io\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.127824 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.127583    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swbjc\" (UniqueName: \"kubernetes.io/projected/59b943fa-86eb-453f-8e2f-63c64ce435ce-kube-api-access-swbjc\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.127824 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.127603    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/59b943fa-86eb-453f-8e2f-63c64ce435ce-host-run-k8s-cni-cncf-io\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.127824 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.127611    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/18ba94f3-1bd7-41f4-a603-0a2654475a66-system-cni-dir\") pod \"multus-additional-cni-plugins-rw58q\" (UID: \"18ba94f3-1bd7-41f4-a603-0a2654475a66\") " pod="openshift-multus/multus-additional-cni-plugins-rw58q"
Jul 23 23:13:32.127824 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.127640    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmffx\" (UniqueName: \"kubernetes.io/projected/18ba94f3-1bd7-41f4-a603-0a2654475a66-kube-api-access-vmffx\") pod \"multus-additional-cni-plugins-rw58q\" (UID: \"18ba94f3-1bd7-41f4-a603-0a2654475a66\") " pod="openshift-multus/multus-additional-cni-plugins-rw58q"
Jul 23 23:13:32.127824 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.127657    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/61f6356e-0fdc-40c3-be10-216d774ff70b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nzjsz\" (UID: \"61f6356e-0fdc-40c3-be10-216d774ff70b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzjsz"
Jul 23 23:13:32.127824 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.127549    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/59b943fa-86eb-453f-8e2f-63c64ce435ce-host-var-lib-cni-bin\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.127824 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.127732    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/61f6356e-0fdc-40c3-be10-216d774ff70b-device-dir\") pod \"aws-ebs-csi-driver-node-nzjsz\" (UID: \"61f6356e-0fdc-40c3-be10-216d774ff70b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzjsz"
Jul 23 23:13:32.127824 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.127765    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dd112f5-f0cc-4e72-917c-33e4abca79cb-metrics-certs\") pod \"network-metrics-daemon-bbv68\" (UID: \"8dd112f5-f0cc-4e72-917c-33e4abca79cb\") " pod="openshift-multus/network-metrics-daemon-bbv68"
Jul 23 23:13:32.127824 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.127782    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnjx6\" (UniqueName: \"kubernetes.io/projected/8dd112f5-f0cc-4e72-917c-33e4abca79cb-kube-api-access-rnjx6\") pod \"network-metrics-daemon-bbv68\" (UID: \"8dd112f5-f0cc-4e72-917c-33e4abca79cb\") " pod="openshift-multus/network-metrics-daemon-bbv68"
Jul 23 23:13:32.127824 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.127800    2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 23 23:13:32.127824 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.127813    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/59b943fa-86eb-453f-8e2f-63c64ce435ce-host-var-lib-cni-multus\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.128497 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.127836    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/18ba94f3-1bd7-41f4-a603-0a2654475a66-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rw58q\" (UID: \"18ba94f3-1bd7-41f4-a603-0a2654475a66\") " pod="openshift-multus/multus-additional-cni-plugins-rw58q"
Jul 23 23:13:32.128497 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.127853    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-run-ovn\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.128497 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.127868    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/18ba94f3-1bd7-41f4-a603-0a2654475a66-cnibin\") pod \"multus-additional-cni-plugins-rw58q\" (UID: \"18ba94f3-1bd7-41f4-a603-0a2654475a66\") " pod="openshift-multus/multus-additional-cni-plugins-rw58q"
Jul 23 23:13:32.128497 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.127892    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/18ba94f3-1bd7-41f4-a603-0a2654475a66-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rw58q\" (UID: \"18ba94f3-1bd7-41f4-a603-0a2654475a66\") " pod="openshift-multus/multus-additional-cni-plugins-rw58q"
Jul 23 23:13:32.128497 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.127912    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-run-openvswitch\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.128497 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.127920    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/59b943fa-86eb-453f-8e2f-63c64ce435ce-host-var-lib-cni-multus\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.128497 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.127930    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-log-socket\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.128497 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.127959    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-env-overrides\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.128497 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.127990    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d2943613-62a6-476e-bec2-12fb3c167e01-serviceca\") pod \"node-ca-ljq8l\" (UID: \"d2943613-62a6-476e-bec2-12fb3c167e01\") " pod="openshift-image-registry/node-ca-ljq8l"
Jul 23 23:13:32.128497 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.128018    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/960f9613-6e9d-4172-a662-b20bc07d8705-agent-certs\") pod \"konnectivity-agent-9jn86\" (UID: \"960f9613-6e9d-4172-a662-b20bc07d8705\") " pod="kube-system/konnectivity-agent-9jn86"
Jul 23 23:13:32.128497 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.128043    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-host-kubelet\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.128497 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.128062    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-host-slash\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.128497 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.128078    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/61f6356e-0fdc-40c3-be10-216d774ff70b-socket-dir\") pod \"aws-ebs-csi-driver-node-nzjsz\" (UID: \"61f6356e-0fdc-40c3-be10-216d774ff70b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzjsz"
Jul 23 23:13:32.128497 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.128095    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/61f6356e-0fdc-40c3-be10-216d774ff70b-registration-dir\") pod \"aws-ebs-csi-driver-node-nzjsz\" (UID: \"61f6356e-0fdc-40c3-be10-216d774ff70b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzjsz"
Jul 23 23:13:32.128497 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.128122    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d1632b98-84f8-4658-af28-6521dd300a59-etc-sysconfig\") pod \"tuned-29zxk\" (UID: \"d1632b98-84f8-4658-af28-6521dd300a59\") " pod="openshift-cluster-node-tuning-operator/tuned-29zxk"
Jul 23 23:13:32.128497 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.128156    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d1632b98-84f8-4658-af28-6521dd300a59-run\") pod \"tuned-29zxk\" (UID: \"d1632b98-84f8-4658-af28-6521dd300a59\") " pod="openshift-cluster-node-tuning-operator/tuned-29zxk"
Jul 23 23:13:32.128497 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.128176    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d1632b98-84f8-4658-af28-6521dd300a59-etc-sysconfig\") pod \"tuned-29zxk\" (UID: \"d1632b98-84f8-4658-af28-6521dd300a59\") " pod="openshift-cluster-node-tuning-operator/tuned-29zxk"
Jul 23 23:13:32.129282 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.128245    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d1632b98-84f8-4658-af28-6521dd300a59-run\") pod \"tuned-29zxk\" (UID: \"d1632b98-84f8-4658-af28-6521dd300a59\") " pod="openshift-cluster-node-tuning-operator/tuned-29zxk"
Jul 23 23:13:32.129282 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.128288    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/59b943fa-86eb-453f-8e2f-63c64ce435ce-hostroot\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.129282 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.128320    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-systemd-units\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.129282 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.128346    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/59b943fa-86eb-453f-8e2f-63c64ce435ce-hostroot\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.129282 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.128349    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-host-run-netns\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.129282 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.128374    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-host-cni-bin\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.129282 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.128398    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b4d53a59-83bb-4725-841c-9f5b55f85ca4-dbus\") pod \"global-pull-secret-syncer-z8d42\" (UID: \"b4d53a59-83bb-4725-841c-9f5b55f85ca4\") " pod="kube-system/global-pull-secret-syncer-z8d42"
Jul 23 23:13:32.129282 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.128407    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d2943613-62a6-476e-bec2-12fb3c167e01-serviceca\") pod \"node-ca-ljq8l\" (UID: \"d2943613-62a6-476e-bec2-12fb3c167e01\") " pod="openshift-image-registry/node-ca-ljq8l"
Jul 23 23:13:32.129282 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.128427    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b4d53a59-83bb-4725-841c-9f5b55f85ca4-original-pull-secret\") pod \"global-pull-secret-syncer-z8d42\" (UID: \"b4d53a59-83bb-4725-841c-9f5b55f85ca4\") " pod="kube-system/global-pull-secret-syncer-z8d42"
Jul 23 23:13:32.129282 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.128455    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bp6zx\" (UniqueName: \"kubernetes.io/projected/737833d3-8a38-4c00-8558-a7bccdc01903-kube-api-access-bp6zx\") pod \"iptables-alerter-v6zh5\" (UID: \"737833d3-8a38-4c00-8558-a7bccdc01903\") " pod="openshift-network-operator/iptables-alerter-v6zh5"
Jul 23 23:13:32.129282 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.128486    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/59b943fa-86eb-453f-8e2f-63c64ce435ce-cni-binary-copy\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.129282 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.128510    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/59b943fa-86eb-453f-8e2f-63c64ce435ce-multus-socket-dir-parent\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.129282 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.128535    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d1632b98-84f8-4658-af28-6521dd300a59-etc-kubernetes\") pod \"tuned-29zxk\" (UID: \"d1632b98-84f8-4658-af28-6521dd300a59\") " pod="openshift-cluster-node-tuning-operator/tuned-29zxk"
Jul 23 23:13:32.129282 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.128560    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/59b943fa-86eb-453f-8e2f-63c64ce435ce-system-cni-dir\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.129282 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.128585    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/59b943fa-86eb-453f-8e2f-63c64ce435ce-host-var-lib-kubelet\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.129282 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.128602    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/59b943fa-86eb-453f-8e2f-63c64ce435ce-multus-socket-dir-parent\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.129282 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.128610    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-etc-openvswitch\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.129895 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.128635    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d1632b98-84f8-4658-af28-6521dd300a59-etc-kubernetes\") pod \"tuned-29zxk\" (UID: \"d1632b98-84f8-4658-af28-6521dd300a59\") " pod="openshift-cluster-node-tuning-operator/tuned-29zxk"
Jul 23 23:13:32.129895 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.128643    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/59b943fa-86eb-453f-8e2f-63c64ce435ce-system-cni-dir\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.129895 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.128694    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/59b943fa-86eb-453f-8e2f-63c64ce435ce-host-var-lib-kubelet\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.129895 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.128727    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-node-log\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.129895 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.128750    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d1632b98-84f8-4658-af28-6521dd300a59-etc-sysctl-d\") pod \"tuned-29zxk\" (UID: \"d1632b98-84f8-4658-af28-6521dd300a59\") " pod="openshift-cluster-node-tuning-operator/tuned-29zxk"
Jul 23 23:13:32.129895 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.128770    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d1632b98-84f8-4658-af28-6521dd300a59-etc-tuned\") pod \"tuned-29zxk\" (UID: \"d1632b98-84f8-4658-af28-6521dd300a59\") " pod="openshift-cluster-node-tuning-operator/tuned-29zxk"
Jul 23 23:13:32.129895 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.128793    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/59b943fa-86eb-453f-8e2f-63c64ce435ce-multus-cni-dir\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.129895 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.128816    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/59b943fa-86eb-453f-8e2f-63c64ce435ce-host-run-netns\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.129895 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.128838    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-run-systemd\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.129895 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.128865    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-host-cni-netd\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.129895 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.128874    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d1632b98-84f8-4658-af28-6521dd300a59-etc-sysctl-d\") pod \"tuned-29zxk\" (UID: \"d1632b98-84f8-4658-af28-6521dd300a59\") " pod="openshift-cluster-node-tuning-operator/tuned-29zxk"
Jul 23 23:13:32.129895 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.128890    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xkk9\" (UniqueName: \"kubernetes.io/projected/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-kube-api-access-7xkk9\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.129895 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.128911    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/59b943fa-86eb-453f-8e2f-63c64ce435ce-host-run-netns\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.129895 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.128919    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/59b943fa-86eb-453f-8e2f-63c64ce435ce-multus-conf-dir\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.129895 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.128960    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/59b943fa-86eb-453f-8e2f-63c64ce435ce-multus-conf-dir\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.129895 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.128999    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-var-lib-openvswitch\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.129895 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.129015    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/59b943fa-86eb-453f-8e2f-63c64ce435ce-multus-cni-dir\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.129895 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.129032    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-host-run-ovn-kubernetes\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.130429 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.129064    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/59b943fa-86eb-453f-8e2f-63c64ce435ce-multus-daemon-config\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.130429 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.129094    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6c30a49c-1943-4650-83f0-adaa3045404e-hosts-file\") pod \"node-resolver-sn5k2\" (UID: \"6c30a49c-1943-4650-83f0-adaa3045404e\") " pod="openshift-dns/node-resolver-sn5k2"
Jul 23 23:13:32.130429 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.129120    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/18ba94f3-1bd7-41f4-a603-0a2654475a66-os-release\") pod \"multus-additional-cni-plugins-rw58q\" (UID: \"18ba94f3-1bd7-41f4-a603-0a2654475a66\") " pod="openshift-multus/multus-additional-cni-plugins-rw58q"
Jul 23 23:13:32.130429 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.129146    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-ovnkube-script-lib\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.130429 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.129173    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d1632b98-84f8-4658-af28-6521dd300a59-etc-sysctl-conf\") pod \"tuned-29zxk\" (UID: \"d1632b98-84f8-4658-af28-6521dd300a59\") " pod="openshift-cluster-node-tuning-operator/tuned-29zxk"
Jul 23 23:13:32.130429 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.129201    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d1632b98-84f8-4658-af28-6521dd300a59-host\") pod \"tuned-29zxk\" (UID: \"d1632b98-84f8-4658-af28-6521dd300a59\") " pod="openshift-cluster-node-tuning-operator/tuned-29zxk"
Jul 23 23:13:32.130429 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.129242    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/59b943fa-86eb-453f-8e2f-63c64ce435ce-cnibin\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.130429 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.129267    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/59b943fa-86eb-453f-8e2f-63c64ce435ce-cni-binary-copy\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.130429 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.129277    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5bk9\" (UniqueName: \"kubernetes.io/projected/f009ab3e-edd9-4bf3-877e-37115c04d13f-kube-api-access-k5bk9\") pod \"network-check-target-5d9sj\" (UID: \"f009ab3e-edd9-4bf3-877e-37115c04d13f\") " pod="openshift-network-diagnostics/network-check-target-5d9sj"
Jul 23 23:13:32.130429 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.129305    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-ovnkube-config\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.130429 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.129332    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b4d53a59-83bb-4725-841c-9f5b55f85ca4-kubelet-config\") pod \"global-pull-secret-syncer-z8d42\" (UID: \"b4d53a59-83bb-4725-841c-9f5b55f85ca4\") " pod="kube-system/global-pull-secret-syncer-z8d42"
Jul 23 23:13:32.130429 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.129340    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d1632b98-84f8-4658-af28-6521dd300a59-host\") pod \"tuned-29zxk\" (UID: \"d1632b98-84f8-4658-af28-6521dd300a59\") " pod="openshift-cluster-node-tuning-operator/tuned-29zxk"
Jul 23 23:13:32.130429 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.129376    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d1632b98-84f8-4658-af28-6521dd300a59-etc-sysctl-conf\") pod \"tuned-29zxk\" (UID: \"d1632b98-84f8-4658-af28-6521dd300a59\") " pod="openshift-cluster-node-tuning-operator/tuned-29zxk"
Jul 23 23:13:32.130429 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.129359    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/61f6356e-0fdc-40c3-be10-216d774ff70b-etc-selinux\") pod \"aws-ebs-csi-driver-node-nzjsz\" (UID: \"61f6356e-0fdc-40c3-be10-216d774ff70b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzjsz"
Jul 23 23:13:32.130429 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.129410    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/59b943fa-86eb-453f-8e2f-63c64ce435ce-cnibin\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.130429 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.129566    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d1632b98-84f8-4658-af28-6521dd300a59-lib-modules\") pod \"tuned-29zxk\" (UID: \"d1632b98-84f8-4658-af28-6521dd300a59\") " pod="openshift-cluster-node-tuning-operator/tuned-29zxk"
Jul 23 23:13:32.130429 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.129590    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d2943613-62a6-476e-bec2-12fb3c167e01-host\") pod \"node-ca-ljq8l\" (UID: \"d2943613-62a6-476e-bec2-12fb3c167e01\") " pod="openshift-image-registry/node-ca-ljq8l"
Jul 23 23:13:32.130960 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.129622    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nsjt\" (UniqueName: \"kubernetes.io/projected/d2943613-62a6-476e-bec2-12fb3c167e01-kube-api-access-7nsjt\") pod \"node-ca-ljq8l\" (UID: \"d2943613-62a6-476e-bec2-12fb3c167e01\") " pod="openshift-image-registry/node-ca-ljq8l"
Jul 23 23:13:32.130960 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.129648    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/59b943fa-86eb-453f-8e2f-63c64ce435ce-host-run-multus-certs\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.130960 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.129675    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d1632b98-84f8-4658-af28-6521dd300a59-etc-systemd\") pod \"tuned-29zxk\" (UID: \"d1632b98-84f8-4658-af28-6521dd300a59\") " pod="openshift-cluster-node-tuning-operator/tuned-29zxk"
Jul 23 23:13:32.130960 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.129723    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/59b943fa-86eb-453f-8e2f-63c64ce435ce-etc-kubernetes\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.130960 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.129750    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/18ba94f3-1bd7-41f4-a603-0a2654475a66-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rw58q\" (UID: \"18ba94f3-1bd7-41f4-a603-0a2654475a66\") " pod="openshift-multus/multus-additional-cni-plugins-rw58q"
Jul 23 23:13:32.130960 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.129779    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/960f9613-6e9d-4172-a662-b20bc07d8705-konnectivity-ca\") pod \"konnectivity-agent-9jn86\" (UID: \"960f9613-6e9d-4172-a662-b20bc07d8705\") " pod="kube-system/konnectivity-agent-9jn86"
Jul 23 23:13:32.130960 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.129804    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d1632b98-84f8-4658-af28-6521dd300a59-etc-modprobe-d\") pod \"tuned-29zxk\" (UID: \"d1632b98-84f8-4658-af28-6521dd300a59\") " pod="openshift-cluster-node-tuning-operator/tuned-29zxk"
Jul 23 23:13:32.130960 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.129831    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lrrw\" (UniqueName: \"kubernetes.io/projected/d1632b98-84f8-4658-af28-6521dd300a59-kube-api-access-5lrrw\") pod \"tuned-29zxk\" (UID: \"d1632b98-84f8-4658-af28-6521dd300a59\") " pod="openshift-cluster-node-tuning-operator/tuned-29zxk"
Jul 23 23:13:32.130960 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.129857    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/59b943fa-86eb-453f-8e2f-63c64ce435ce-os-release\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.130960 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.129883    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.130960 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.129911    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d1632b98-84f8-4658-af28-6521dd300a59-sys\") pod \"tuned-29zxk\" (UID: \"d1632b98-84f8-4658-af28-6521dd300a59\") " pod="openshift-cluster-node-tuning-operator/tuned-29zxk"
Jul 23 23:13:32.130960 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.129939    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fczdz\" (UniqueName: \"kubernetes.io/projected/6c30a49c-1943-4650-83f0-adaa3045404e-kube-api-access-fczdz\") pod \"node-resolver-sn5k2\" (UID: \"6c30a49c-1943-4650-83f0-adaa3045404e\") " pod="openshift-dns/node-resolver-sn5k2"
Jul 23 23:13:32.130960 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.129966    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/18ba94f3-1bd7-41f4-a603-0a2654475a66-cni-binary-copy\") pod \"multus-additional-cni-plugins-rw58q\" (UID: \"18ba94f3-1bd7-41f4-a603-0a2654475a66\") " pod="openshift-multus/multus-additional-cni-plugins-rw58q"
Jul 23 23:13:32.130960 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.129993    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/737833d3-8a38-4c00-8558-a7bccdc01903-iptables-alerter-script\") pod \"iptables-alerter-v6zh5\" (UID: \"737833d3-8a38-4c00-8558-a7bccdc01903\") " pod="openshift-network-operator/iptables-alerter-v6zh5"
Jul 23 23:13:32.130960 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.130019    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/737833d3-8a38-4c00-8558-a7bccdc01903-host-slash\") pod \"iptables-alerter-v6zh5\" (UID: \"737833d3-8a38-4c00-8558-a7bccdc01903\") " pod="openshift-network-operator/iptables-alerter-v6zh5"
Jul 23 23:13:32.130960 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.130042    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/61f6356e-0fdc-40c3-be10-216d774ff70b-sys-fs\") pod \"aws-ebs-csi-driver-node-nzjsz\" (UID: \"61f6356e-0fdc-40c3-be10-216d774ff70b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzjsz"
Jul 23 23:13:32.130960 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.130205    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/59b943fa-86eb-453f-8e2f-63c64ce435ce-multus-daemon-config\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.131466 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.130203    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d1632b98-84f8-4658-af28-6521dd300a59-lib-modules\") pod \"tuned-29zxk\" (UID: \"d1632b98-84f8-4658-af28-6521dd300a59\") " pod="openshift-cluster-node-tuning-operator/tuned-29zxk"
Jul 23 23:13:32.131466 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.130289    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d1632b98-84f8-4658-af28-6521dd300a59-etc-modprobe-d\") pod \"tuned-29zxk\" (UID: \"d1632b98-84f8-4658-af28-6521dd300a59\") " pod="openshift-cluster-node-tuning-operator/tuned-29zxk"
Jul 23 23:13:32.131466 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.130319    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d2943613-62a6-476e-bec2-12fb3c167e01-host\") pod \"node-ca-ljq8l\" (UID: \"d2943613-62a6-476e-bec2-12fb3c167e01\") " pod="openshift-image-registry/node-ca-ljq8l"
Jul 23 23:13:32.131466 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.130474    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/59b943fa-86eb-453f-8e2f-63c64ce435ce-host-run-multus-certs\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.131466 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.130517    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d1632b98-84f8-4658-af28-6521dd300a59-etc-systemd\") pod \"tuned-29zxk\" (UID: \"d1632b98-84f8-4658-af28-6521dd300a59\") " pod="openshift-cluster-node-tuning-operator/tuned-29zxk"
Jul 23 23:13:32.131466 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.130584    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d1632b98-84f8-4658-af28-6521dd300a59-sys\") pod \"tuned-29zxk\" (UID: \"d1632b98-84f8-4658-af28-6521dd300a59\") " pod="openshift-cluster-node-tuning-operator/tuned-29zxk"
Jul 23 23:13:32.131466 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.130589    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/59b943fa-86eb-453f-8e2f-63c64ce435ce-os-release\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.131466 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.130625    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/59b943fa-86eb-453f-8e2f-63c64ce435ce-etc-kubernetes\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.131466 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.131305    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d1632b98-84f8-4658-af28-6521dd300a59-tmp\") pod \"tuned-29zxk\" (UID: \"d1632b98-84f8-4658-af28-6521dd300a59\") " pod="openshift-cluster-node-tuning-operator/tuned-29zxk"
Jul 23 23:13:32.131466 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.131345    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d1632b98-84f8-4658-af28-6521dd300a59-etc-tuned\") pod \"tuned-29zxk\" (UID: \"d1632b98-84f8-4658-af28-6521dd300a59\") " pod="openshift-cluster-node-tuning-operator/tuned-29zxk"
Jul 23 23:13:32.136584 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.136565    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-swbjc\" (UniqueName: \"kubernetes.io/projected/59b943fa-86eb-453f-8e2f-63c64ce435ce-kube-api-access-swbjc\") pod \"multus-f66wp\" (UID: \"59b943fa-86eb-453f-8e2f-63c64ce435ce\") " pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.142306 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.142272    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lrrw\" (UniqueName: \"kubernetes.io/projected/d1632b98-84f8-4658-af28-6521dd300a59-kube-api-access-5lrrw\") pod \"tuned-29zxk\" (UID: \"d1632b98-84f8-4658-af28-6521dd300a59\") " pod="openshift-cluster-node-tuning-operator/tuned-29zxk"
Jul 23 23:13:32.142580 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.142556    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nsjt\" (UniqueName: \"kubernetes.io/projected/d2943613-62a6-476e-bec2-12fb3c167e01-kube-api-access-7nsjt\") pod \"node-ca-ljq8l\" (UID: \"d2943613-62a6-476e-bec2-12fb3c167e01\") " pod="openshift-image-registry/node-ca-ljq8l"
Jul 23 23:13:32.162953 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.162930    2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 23:13:32.230683 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.230643    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-systemd-units\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.230901 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.230698    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-host-run-netns\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.230901 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.230764    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-host-run-netns\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.230901 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.230799    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-host-cni-bin\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.230901 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.230823    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b4d53a59-83bb-4725-841c-9f5b55f85ca4-dbus\") pod \"global-pull-secret-syncer-z8d42\" (UID: \"b4d53a59-83bb-4725-841c-9f5b55f85ca4\") " pod="kube-system/global-pull-secret-syncer-z8d42"
Jul 23 23:13:32.230901 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.230829    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-host-cni-bin\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.230901 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.230769    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-systemd-units\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.230901 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.230846    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b4d53a59-83bb-4725-841c-9f5b55f85ca4-original-pull-secret\") pod \"global-pull-secret-syncer-z8d42\" (UID: \"b4d53a59-83bb-4725-841c-9f5b55f85ca4\") " pod="kube-system/global-pull-secret-syncer-z8d42"
Jul 23 23:13:32.230901 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.230881    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bp6zx\" (UniqueName: \"kubernetes.io/projected/737833d3-8a38-4c00-8558-a7bccdc01903-kube-api-access-bp6zx\") pod \"iptables-alerter-v6zh5\" (UID: \"737833d3-8a38-4c00-8558-a7bccdc01903\") " pod="openshift-network-operator/iptables-alerter-v6zh5"
Jul 23 23:13:32.230901 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.230904    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-etc-openvswitch\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.231386 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.230920    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-node-log\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.231386 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:32.230939    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 23:13:32.231386 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.230949    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-run-systemd\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.231386 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.230949    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b4d53a59-83bb-4725-841c-9f5b55f85ca4-dbus\") pod \"global-pull-secret-syncer-z8d42\" (UID: \"b4d53a59-83bb-4725-841c-9f5b55f85ca4\") " pod="kube-system/global-pull-secret-syncer-z8d42"
Jul 23 23:13:32.231386 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.230974    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-host-cni-netd\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.231386 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:32.231011    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4d53a59-83bb-4725-841c-9f5b55f85ca4-original-pull-secret podName:b4d53a59-83bb-4725-841c-9f5b55f85ca4 nodeName:}" failed. No retries permitted until 2026-07-23 23:13:32.730984152 +0000 UTC m=+2.100509794 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b4d53a59-83bb-4725-841c-9f5b55f85ca4-original-pull-secret") pod "global-pull-secret-syncer-z8d42" (UID: "b4d53a59-83bb-4725-841c-9f5b55f85ca4") : object "kube-system"/"original-pull-secret" not registered
Jul 23 23:13:32.231386 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.231017    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-host-cni-netd\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.231386 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.231013    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-run-systemd\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.231386 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.231031    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xkk9\" (UniqueName: \"kubernetes.io/projected/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-kube-api-access-7xkk9\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.231386 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.231014    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-etc-openvswitch\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.231386 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.231008    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-node-log\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.231386 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.231070    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-var-lib-openvswitch\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.231386 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.231109    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-var-lib-openvswitch\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.231386 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.231139    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-host-run-ovn-kubernetes\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.231386 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.231208    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6c30a49c-1943-4650-83f0-adaa3045404e-hosts-file\") pod \"node-resolver-sn5k2\" (UID: \"6c30a49c-1943-4650-83f0-adaa3045404e\") " pod="openshift-dns/node-resolver-sn5k2"
Jul 23 23:13:32.231386 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.231219    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-host-run-ovn-kubernetes\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.231386 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.231236    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/18ba94f3-1bd7-41f4-a603-0a2654475a66-os-release\") pod \"multus-additional-cni-plugins-rw58q\" (UID: \"18ba94f3-1bd7-41f4-a603-0a2654475a66\") " pod="openshift-multus/multus-additional-cni-plugins-rw58q"
Jul 23 23:13:32.232311 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.231266    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-ovnkube-script-lib\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.232311 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.231295    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5bk9\" (UniqueName: \"kubernetes.io/projected/f009ab3e-edd9-4bf3-877e-37115c04d13f-kube-api-access-k5bk9\") pod \"network-check-target-5d9sj\" (UID: \"f009ab3e-edd9-4bf3-877e-37115c04d13f\") " pod="openshift-network-diagnostics/network-check-target-5d9sj"
Jul 23 23:13:32.232311 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.231304    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6c30a49c-1943-4650-83f0-adaa3045404e-hosts-file\") pod \"node-resolver-sn5k2\" (UID: \"6c30a49c-1943-4650-83f0-adaa3045404e\") " pod="openshift-dns/node-resolver-sn5k2"
Jul 23 23:13:32.232311 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.231327    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/18ba94f3-1bd7-41f4-a603-0a2654475a66-os-release\") pod \"multus-additional-cni-plugins-rw58q\" (UID: \"18ba94f3-1bd7-41f4-a603-0a2654475a66\") " pod="openshift-multus/multus-additional-cni-plugins-rw58q"
Jul 23 23:13:32.232311 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.231338    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-ovnkube-config\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.232311 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.231365    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b4d53a59-83bb-4725-841c-9f5b55f85ca4-kubelet-config\") pod \"global-pull-secret-syncer-z8d42\" (UID: \"b4d53a59-83bb-4725-841c-9f5b55f85ca4\") " pod="kube-system/global-pull-secret-syncer-z8d42"
Jul 23 23:13:32.232311 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.231390    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/61f6356e-0fdc-40c3-be10-216d774ff70b-etc-selinux\") pod \"aws-ebs-csi-driver-node-nzjsz\" (UID: \"61f6356e-0fdc-40c3-be10-216d774ff70b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzjsz"
Jul 23 23:13:32.232311 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.231420    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/18ba94f3-1bd7-41f4-a603-0a2654475a66-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rw58q\" (UID: \"18ba94f3-1bd7-41f4-a603-0a2654475a66\") " pod="openshift-multus/multus-additional-cni-plugins-rw58q"
Jul 23 23:13:32.232311 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.231450    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/960f9613-6e9d-4172-a662-b20bc07d8705-konnectivity-ca\") pod \"konnectivity-agent-9jn86\" (UID: \"960f9613-6e9d-4172-a662-b20bc07d8705\") " pod="kube-system/konnectivity-agent-9jn86"
Jul 23 23:13:32.232311 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.231481    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.232311 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.231509    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fczdz\" (UniqueName: \"kubernetes.io/projected/6c30a49c-1943-4650-83f0-adaa3045404e-kube-api-access-fczdz\") pod \"node-resolver-sn5k2\" (UID: \"6c30a49c-1943-4650-83f0-adaa3045404e\") " pod="openshift-dns/node-resolver-sn5k2"
Jul 23 23:13:32.232311 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.231514    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/61f6356e-0fdc-40c3-be10-216d774ff70b-etc-selinux\") pod \"aws-ebs-csi-driver-node-nzjsz\" (UID: \"61f6356e-0fdc-40c3-be10-216d774ff70b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzjsz"
Jul 23 23:13:32.232311 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.231537    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/18ba94f3-1bd7-41f4-a603-0a2654475a66-cni-binary-copy\") pod \"multus-additional-cni-plugins-rw58q\" (UID: \"18ba94f3-1bd7-41f4-a603-0a2654475a66\") " pod="openshift-multus/multus-additional-cni-plugins-rw58q"
Jul 23 23:13:32.232311 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.231566    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/737833d3-8a38-4c00-8558-a7bccdc01903-iptables-alerter-script\") pod \"iptables-alerter-v6zh5\" (UID: \"737833d3-8a38-4c00-8558-a7bccdc01903\") " pod="openshift-network-operator/iptables-alerter-v6zh5"
Jul 23 23:13:32.232311 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.231583    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.232311 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.231596    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/737833d3-8a38-4c00-8558-a7bccdc01903-host-slash\") pod \"iptables-alerter-v6zh5\" (UID: \"737833d3-8a38-4c00-8558-a7bccdc01903\") " pod="openshift-network-operator/iptables-alerter-v6zh5"
Jul 23 23:13:32.232311 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.231612    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b4d53a59-83bb-4725-841c-9f5b55f85ca4-kubelet-config\") pod \"global-pull-secret-syncer-z8d42\" (UID: \"b4d53a59-83bb-4725-841c-9f5b55f85ca4\") " pod="kube-system/global-pull-secret-syncer-z8d42"
Jul 23 23:13:32.233145 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.231641    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/61f6356e-0fdc-40c3-be10-216d774ff70b-sys-fs\") pod \"aws-ebs-csi-driver-node-nzjsz\" (UID: \"61f6356e-0fdc-40c3-be10-216d774ff70b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzjsz"
Jul 23 23:13:32.233145 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.231690    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c30a49c-1943-4650-83f0-adaa3045404e-tmp-dir\") pod \"node-resolver-sn5k2\" (UID: \"6c30a49c-1943-4650-83f0-adaa3045404e\") " pod="openshift-dns/node-resolver-sn5k2"
Jul 23 23:13:32.233145 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.231719    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-ovn-node-metrics-cert\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.233145 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.231645    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/737833d3-8a38-4c00-8558-a7bccdc01903-host-slash\") pod \"iptables-alerter-v6zh5\" (UID: \"737833d3-8a38-4c00-8558-a7bccdc01903\") " pod="openshift-network-operator/iptables-alerter-v6zh5"
Jul 23 23:13:32.233145 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.231749    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvq8j\" (UniqueName: \"kubernetes.io/projected/61f6356e-0fdc-40c3-be10-216d774ff70b-kube-api-access-lvq8j\") pod \"aws-ebs-csi-driver-node-nzjsz\" (UID: \"61f6356e-0fdc-40c3-be10-216d774ff70b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzjsz"
Jul 23 23:13:32.233145 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.231802    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/18ba94f3-1bd7-41f4-a603-0a2654475a66-system-cni-dir\") pod \"multus-additional-cni-plugins-rw58q\" (UID: \"18ba94f3-1bd7-41f4-a603-0a2654475a66\") " pod="openshift-multus/multus-additional-cni-plugins-rw58q"
Jul 23 23:13:32.233145 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.231854    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/18ba94f3-1bd7-41f4-a603-0a2654475a66-system-cni-dir\") pod \"multus-additional-cni-plugins-rw58q\" (UID: \"18ba94f3-1bd7-41f4-a603-0a2654475a66\") " pod="openshift-multus/multus-additional-cni-plugins-rw58q"
Jul 23 23:13:32.233145 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.231872    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/61f6356e-0fdc-40c3-be10-216d774ff70b-sys-fs\") pod \"aws-ebs-csi-driver-node-nzjsz\" (UID: \"61f6356e-0fdc-40c3-be10-216d774ff70b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzjsz"
Jul 23 23:13:32.233145 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.231908    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmffx\" (UniqueName: \"kubernetes.io/projected/18ba94f3-1bd7-41f4-a603-0a2654475a66-kube-api-access-vmffx\") pod \"multus-additional-cni-plugins-rw58q\" (UID: \"18ba94f3-1bd7-41f4-a603-0a2654475a66\") " pod="openshift-multus/multus-additional-cni-plugins-rw58q"
Jul 23 23:13:32.233145 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.231938    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/61f6356e-0fdc-40c3-be10-216d774ff70b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nzjsz\" (UID: \"61f6356e-0fdc-40c3-be10-216d774ff70b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzjsz"
Jul 23 23:13:32.233145 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.231971    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/61f6356e-0fdc-40c3-be10-216d774ff70b-device-dir\") pod \"aws-ebs-csi-driver-node-nzjsz\" (UID: \"61f6356e-0fdc-40c3-be10-216d774ff70b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzjsz"
Jul 23 23:13:32.233145 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.231997    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dd112f5-f0cc-4e72-917c-33e4abca79cb-metrics-certs\") pod \"network-metrics-daemon-bbv68\" (UID: \"8dd112f5-f0cc-4e72-917c-33e4abca79cb\") " pod="openshift-multus/network-metrics-daemon-bbv68"
Jul 23 23:13:32.233145 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.232029    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnjx6\" (UniqueName: \"kubernetes.io/projected/8dd112f5-f0cc-4e72-917c-33e4abca79cb-kube-api-access-rnjx6\") pod \"network-metrics-daemon-bbv68\" (UID: \"8dd112f5-f0cc-4e72-917c-33e4abca79cb\") " pod="openshift-multus/network-metrics-daemon-bbv68"
Jul 23 23:13:32.233145 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.232059    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/18ba94f3-1bd7-41f4-a603-0a2654475a66-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rw58q\" (UID: \"18ba94f3-1bd7-41f4-a603-0a2654475a66\") " pod="openshift-multus/multus-additional-cni-plugins-rw58q"
Jul 23 23:13:32.233145 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.232083    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-run-ovn\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.233145 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.232107    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/18ba94f3-1bd7-41f4-a603-0a2654475a66-cnibin\") pod \"multus-additional-cni-plugins-rw58q\" (UID: \"18ba94f3-1bd7-41f4-a603-0a2654475a66\") " pod="openshift-multus/multus-additional-cni-plugins-rw58q"
Jul 23 23:13:32.233145 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.232122    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-ovnkube-script-lib\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.233796 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.232133    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/18ba94f3-1bd7-41f4-a603-0a2654475a66-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rw58q\" (UID: \"18ba94f3-1bd7-41f4-a603-0a2654475a66\") " pod="openshift-multus/multus-additional-cni-plugins-rw58q"
Jul 23 23:13:32.233796 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.232125    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-ovnkube-config\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.233796 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.232173    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/960f9613-6e9d-4172-a662-b20bc07d8705-konnectivity-ca\") pod \"konnectivity-agent-9jn86\" (UID: \"960f9613-6e9d-4172-a662-b20bc07d8705\") " pod="kube-system/konnectivity-agent-9jn86"
Jul 23 23:13:32.233796 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.232202    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c30a49c-1943-4650-83f0-adaa3045404e-tmp-dir\") pod \"node-resolver-sn5k2\" (UID: \"6c30a49c-1943-4650-83f0-adaa3045404e\") " pod="openshift-dns/node-resolver-sn5k2"
Jul 23 23:13:32.233796 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:32.232221    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 23:13:32.233796 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.232236    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/61f6356e-0fdc-40c3-be10-216d774ff70b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nzjsz\" (UID: \"61f6356e-0fdc-40c3-be10-216d774ff70b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzjsz"
Jul 23 23:13:32.233796 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:32.232281    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dd112f5-f0cc-4e72-917c-33e4abca79cb-metrics-certs podName:8dd112f5-f0cc-4e72-917c-33e4abca79cb nodeName:}" failed. No retries permitted until 2026-07-23 23:13:32.732264674 +0000 UTC m=+2.101790313 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8dd112f5-f0cc-4e72-917c-33e4abca79cb-metrics-certs") pod "network-metrics-daemon-bbv68" (UID: "8dd112f5-f0cc-4e72-917c-33e4abca79cb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 23:13:32.233796 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.232331    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/61f6356e-0fdc-40c3-be10-216d774ff70b-device-dir\") pod \"aws-ebs-csi-driver-node-nzjsz\" (UID: \"61f6356e-0fdc-40c3-be10-216d774ff70b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzjsz"
Jul 23 23:13:32.233796 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.232344    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/18ba94f3-1bd7-41f4-a603-0a2654475a66-cni-binary-copy\") pod \"multus-additional-cni-plugins-rw58q\" (UID: \"18ba94f3-1bd7-41f4-a603-0a2654475a66\") " pod="openshift-multus/multus-additional-cni-plugins-rw58q"
Jul 23 23:13:32.233796 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.232412    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-run-ovn\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.233796 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.232458    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/18ba94f3-1bd7-41f4-a603-0a2654475a66-cnibin\") pod \"multus-additional-cni-plugins-rw58q\" (UID: \"18ba94f3-1bd7-41f4-a603-0a2654475a66\") " pod="openshift-multus/multus-additional-cni-plugins-rw58q"
Jul 23 23:13:32.233796 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.232497    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-run-openvswitch\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.233796 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.232526    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-log-socket\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.233796 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.232551    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-env-overrides\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.233796 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.232577    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/960f9613-6e9d-4172-a662-b20bc07d8705-agent-certs\") pod \"konnectivity-agent-9jn86\" (UID: \"960f9613-6e9d-4172-a662-b20bc07d8705\") " pod="kube-system/konnectivity-agent-9jn86"
Jul 23 23:13:32.233796 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.232582    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-run-openvswitch\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.233796 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.232590    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-log-socket\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.234442 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.232601    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-host-kubelet\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.234442 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.232627    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-host-slash\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.234442 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.232652    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/61f6356e-0fdc-40c3-be10-216d774ff70b-socket-dir\") pod \"aws-ebs-csi-driver-node-nzjsz\" (UID: \"61f6356e-0fdc-40c3-be10-216d774ff70b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzjsz"
Jul 23 23:13:32.234442 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.232677    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/61f6356e-0fdc-40c3-be10-216d774ff70b-registration-dir\") pod \"aws-ebs-csi-driver-node-nzjsz\" (UID: \"61f6356e-0fdc-40c3-be10-216d774ff70b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzjsz"
Jul 23 23:13:32.234442 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.232704    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/737833d3-8a38-4c00-8558-a7bccdc01903-iptables-alerter-script\") pod \"iptables-alerter-v6zh5\" (UID: \"737833d3-8a38-4c00-8558-a7bccdc01903\") " pod="openshift-network-operator/iptables-alerter-v6zh5"
Jul 23 23:13:32.234442 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.232775    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/61f6356e-0fdc-40c3-be10-216d774ff70b-registration-dir\") pod \"aws-ebs-csi-driver-node-nzjsz\" (UID: \"61f6356e-0fdc-40c3-be10-216d774ff70b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzjsz"
Jul 23 23:13:32.234442 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.232777    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-host-kubelet\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.234442 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.232834    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-host-slash\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.234442 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.232846    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/18ba94f3-1bd7-41f4-a603-0a2654475a66-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rw58q\" (UID: \"18ba94f3-1bd7-41f4-a603-0a2654475a66\") " pod="openshift-multus/multus-additional-cni-plugins-rw58q"
Jul 23 23:13:32.234442 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.232885    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/61f6356e-0fdc-40c3-be10-216d774ff70b-socket-dir\") pod \"aws-ebs-csi-driver-node-nzjsz\" (UID: \"61f6356e-0fdc-40c3-be10-216d774ff70b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzjsz"
Jul 23 23:13:32.234442 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.233015    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/18ba94f3-1bd7-41f4-a603-0a2654475a66-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rw58q\" (UID: \"18ba94f3-1bd7-41f4-a603-0a2654475a66\") " pod="openshift-multus/multus-additional-cni-plugins-rw58q"
Jul 23 23:13:32.234442 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.233054    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-env-overrides\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.234442 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.233091    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/18ba94f3-1bd7-41f4-a603-0a2654475a66-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rw58q\" (UID: \"18ba94f3-1bd7-41f4-a603-0a2654475a66\") " pod="openshift-multus/multus-additional-cni-plugins-rw58q"
Jul 23 23:13:32.235077 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.234812    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-ovn-node-metrics-cert\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.235391 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.235372    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/960f9613-6e9d-4172-a662-b20bc07d8705-agent-certs\") pod \"konnectivity-agent-9jn86\" (UID: \"960f9613-6e9d-4172-a662-b20bc07d8705\") " pod="kube-system/konnectivity-agent-9jn86"
Jul 23 23:13:32.241804 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:32.241773    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 23:13:32.241804 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:32.241811    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 23:13:32.242076 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:32.241826    2578 projected.go:196] Error preparing data for projected volume kube-api-access-k5bk9 for pod openshift-network-diagnostics/network-check-target-5d9sj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 23:13:32.242076 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:32.241916    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f009ab3e-edd9-4bf3-877e-37115c04d13f-kube-api-access-k5bk9 podName:f009ab3e-edd9-4bf3-877e-37115c04d13f nodeName:}" failed. No retries permitted until 2026-07-23 23:13:32.741895706 +0000 UTC m=+2.111421374 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-k5bk9" (UniqueName: "kubernetes.io/projected/f009ab3e-edd9-4bf3-877e-37115c04d13f-kube-api-access-k5bk9") pod "network-check-target-5d9sj" (UID: "f009ab3e-edd9-4bf3-877e-37115c04d13f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 23:13:32.243540 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.243516    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvq8j\" (UniqueName: \"kubernetes.io/projected/61f6356e-0fdc-40c3-be10-216d774ff70b-kube-api-access-lvq8j\") pod \"aws-ebs-csi-driver-node-nzjsz\" (UID: \"61f6356e-0fdc-40c3-be10-216d774ff70b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzjsz"
Jul 23 23:13:32.243860 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.243836    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fczdz\" (UniqueName: \"kubernetes.io/projected/6c30a49c-1943-4650-83f0-adaa3045404e-kube-api-access-fczdz\") pod \"node-resolver-sn5k2\" (UID: \"6c30a49c-1943-4650-83f0-adaa3045404e\") " pod="openshift-dns/node-resolver-sn5k2"
Jul 23 23:13:32.244015 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.243994    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmffx\" (UniqueName: \"kubernetes.io/projected/18ba94f3-1bd7-41f4-a603-0a2654475a66-kube-api-access-vmffx\") pod \"multus-additional-cni-plugins-rw58q\" (UID: \"18ba94f3-1bd7-41f4-a603-0a2654475a66\") " pod="openshift-multus/multus-additional-cni-plugins-rw58q"
Jul 23 23:13:32.244412 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.244389    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bp6zx\" (UniqueName: \"kubernetes.io/projected/737833d3-8a38-4c00-8558-a7bccdc01903-kube-api-access-bp6zx\") pod \"iptables-alerter-v6zh5\" (UID: \"737833d3-8a38-4c00-8558-a7bccdc01903\") " pod="openshift-network-operator/iptables-alerter-v6zh5"
Jul 23 23:13:32.244519 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.244499    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnjx6\" (UniqueName: \"kubernetes.io/projected/8dd112f5-f0cc-4e72-917c-33e4abca79cb-kube-api-access-rnjx6\") pod \"network-metrics-daemon-bbv68\" (UID: \"8dd112f5-f0cc-4e72-917c-33e4abca79cb\") " pod="openshift-multus/network-metrics-daemon-bbv68"
Jul 23 23:13:32.244661 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.244643    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xkk9\" (UniqueName: \"kubernetes.io/projected/ca6b871c-0d2a-44cb-832d-2fd4f7e6395b-kube-api-access-7xkk9\") pod \"ovnkube-node-mx6j5\" (UID: \"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b\") " pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.266018 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:32.265968    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef9bd6a40b72bafbfb43200d1ac0a01d.slice/crio-9dd22b22c4fcec23b58031406e8b6fce47c78a00aaf3a24ebfea07c09837b3ef WatchSource:0}: Error finding container 9dd22b22c4fcec23b58031406e8b6fce47c78a00aaf3a24ebfea07c09837b3ef: Status 404 returned error can't find the container with id 9dd22b22c4fcec23b58031406e8b6fce47c78a00aaf3a24ebfea07c09837b3ef
Jul 23 23:13:32.270553 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.270536    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 23:13:32.321165 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:32.321132    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2a7eb7430412c2da89feeebacf09842.slice/crio-052cd3055f9c3f90da843d8ef448518d222e0bea8da21139dc0048a8ca59d20d WatchSource:0}: Error finding container 052cd3055f9c3f90da843d8ef448518d222e0bea8da21139dc0048a8ca59d20d: Status 404 returned error can't find the container with id 052cd3055f9c3f90da843d8ef448518d222e0bea8da21139dc0048a8ca59d20d
Jul 23 23:13:32.345740 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.345622    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-29zxk"
Jul 23 23:13:32.353569 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.353547    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ljq8l"
Jul 23 23:13:32.356891 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:32.356864    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1632b98_84f8_4658_af28_6521dd300a59.slice/crio-1b18034b76203bbd37a6a8b99c865186f68c45bc09dadb2b77e62af1b80883cb WatchSource:0}: Error finding container 1b18034b76203bbd37a6a8b99c865186f68c45bc09dadb2b77e62af1b80883cb: Status 404 returned error can't find the container with id 1b18034b76203bbd37a6a8b99c865186f68c45bc09dadb2b77e62af1b80883cb
Jul 23 23:13:32.359876 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.359843    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-f66wp"
Jul 23 23:13:32.363761 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:32.363739    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2943613_62a6_476e_bec2_12fb3c167e01.slice/crio-7c9a3a5b71f7bf8fdf2f9c67e2765fbf721dc5ba936c04dddeeff108e875bb6f WatchSource:0}: Error finding container 7c9a3a5b71f7bf8fdf2f9c67e2765fbf721dc5ba936c04dddeeff108e875bb6f: Status 404 returned error can't find the container with id 7c9a3a5b71f7bf8fdf2f9c67e2765fbf721dc5ba936c04dddeeff108e875bb6f
Jul 23 23:13:32.369006 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:32.368981    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59b943fa_86eb_453f_8e2f_63c64ce435ce.slice/crio-9fef8f02f7a3cb82f447df5eb2509bc7e156aa1e99c2e9701e6df89112c80900 WatchSource:0}: Error finding container 9fef8f02f7a3cb82f447df5eb2509bc7e156aa1e99c2e9701e6df89112c80900: Status 404 returned error can't find the container with id 9fef8f02f7a3cb82f447df5eb2509bc7e156aa1e99c2e9701e6df89112c80900
Jul 23 23:13:32.379862 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.379834    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9jn86"
Jul 23 23:13:32.388412 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:32.388383    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod960f9613_6e9d_4172_a662_b20bc07d8705.slice/crio-123e646f7db0d6df8598328afd5f1bfb411dfdef62eeae6a3dc37ca0bcf5b9db WatchSource:0}: Error finding container 123e646f7db0d6df8598328afd5f1bfb411dfdef62eeae6a3dc37ca0bcf5b9db: Status 404 returned error can't find the container with id 123e646f7db0d6df8598328afd5f1bfb411dfdef62eeae6a3dc37ca0bcf5b9db
Jul 23 23:13:32.404203 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.404169    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sn5k2"
Jul 23 23:13:32.412608 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:32.412582    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c30a49c_1943_4650_83f0_adaa3045404e.slice/crio-571e1f53950e26bd75822d65c747ba336d2c86247c1d38e93b6324c533217f0d WatchSource:0}: Error finding container 571e1f53950e26bd75822d65c747ba336d2c86247c1d38e93b6324c533217f0d: Status 404 returned error can't find the container with id 571e1f53950e26bd75822d65c747ba336d2c86247c1d38e93b6324c533217f0d
Jul 23 23:13:32.425927 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.425897    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rw58q"
Jul 23 23:13:32.434528 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:32.434503    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18ba94f3_1bd7_41f4_a603_0a2654475a66.slice/crio-84e41283ab378212df5f4e09e06538b7d46a010fc2fe18eaf87d68ebd1e19936 WatchSource:0}: Error finding container 84e41283ab378212df5f4e09e06538b7d46a010fc2fe18eaf87d68ebd1e19936: Status 404 returned error can't find the container with id 84e41283ab378212df5f4e09e06538b7d46a010fc2fe18eaf87d68ebd1e19936
Jul 23 23:13:32.445103 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.445078    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-v6zh5"
Jul 23 23:13:32.453519 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:32.453490    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod737833d3_8a38_4c00_8558_a7bccdc01903.slice/crio-3bfb65d4c9004ccff559ebd75647ac3692665cd0ad1da8547d7c99cabd2cf120 WatchSource:0}: Error finding container 3bfb65d4c9004ccff559ebd75647ac3692665cd0ad1da8547d7c99cabd2cf120: Status 404 returned error can't find the container with id 3bfb65d4c9004ccff559ebd75647ac3692665cd0ad1da8547d7c99cabd2cf120
Jul 23 23:13:32.463190 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.463164    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:13:32.470475 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.470447    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzjsz"
Jul 23 23:13:32.471193 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:32.471169    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca6b871c_0d2a_44cb_832d_2fd4f7e6395b.slice/crio-3b37a775521bd1b3ffa1e86e0d7e8e040fb789d9fd51db600a46f2e66e7597b5 WatchSource:0}: Error finding container 3b37a775521bd1b3ffa1e86e0d7e8e040fb789d9fd51db600a46f2e66e7597b5: Status 404 returned error can't find the container with id 3b37a775521bd1b3ffa1e86e0d7e8e040fb789d9fd51db600a46f2e66e7597b5
Jul 23 23:13:32.479628 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:13:32.479598    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61f6356e_0fdc_40c3_be10_216d774ff70b.slice/crio-4d1a6c2c3028bf048bd743e33e25060773f6645f4a2ed7d374333b7a36344bdc WatchSource:0}: Error finding container 4d1a6c2c3028bf048bd743e33e25060773f6645f4a2ed7d374333b7a36344bdc: Status 404 returned error can't find the container with id 4d1a6c2c3028bf048bd743e33e25060773f6645f4a2ed7d374333b7a36344bdc
Jul 23 23:13:32.737221 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.737122    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dd112f5-f0cc-4e72-917c-33e4abca79cb-metrics-certs\") pod \"network-metrics-daemon-bbv68\" (UID: \"8dd112f5-f0cc-4e72-917c-33e4abca79cb\") " pod="openshift-multus/network-metrics-daemon-bbv68"
Jul 23 23:13:32.737221 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.737169    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b4d53a59-83bb-4725-841c-9f5b55f85ca4-original-pull-secret\") pod \"global-pull-secret-syncer-z8d42\" (UID: \"b4d53a59-83bb-4725-841c-9f5b55f85ca4\") " pod="kube-system/global-pull-secret-syncer-z8d42"
Jul 23 23:13:32.737442 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:32.737286    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 23:13:32.737442 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:32.737286    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 23:13:32.737442 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:32.737355    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dd112f5-f0cc-4e72-917c-33e4abca79cb-metrics-certs podName:8dd112f5-f0cc-4e72-917c-33e4abca79cb nodeName:}" failed. No retries permitted until 2026-07-23 23:13:33.737336495 +0000 UTC m=+3.106862133 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8dd112f5-f0cc-4e72-917c-33e4abca79cb-metrics-certs") pod "network-metrics-daemon-bbv68" (UID: "8dd112f5-f0cc-4e72-917c-33e4abca79cb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 23:13:32.737442 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:32.737374    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4d53a59-83bb-4725-841c-9f5b55f85ca4-original-pull-secret podName:b4d53a59-83bb-4725-841c-9f5b55f85ca4 nodeName:}" failed. No retries permitted until 2026-07-23 23:13:33.737365152 +0000 UTC m=+3.106890797 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b4d53a59-83bb-4725-841c-9f5b55f85ca4-original-pull-secret") pod "global-pull-secret-syncer-z8d42" (UID: "b4d53a59-83bb-4725-841c-9f5b55f85ca4") : object "kube-system"/"original-pull-secret" not registered
Jul 23 23:13:32.772846 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.772809    2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 23:13:32.838478 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:32.838429    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5bk9\" (UniqueName: \"kubernetes.io/projected/f009ab3e-edd9-4bf3-877e-37115c04d13f-kube-api-access-k5bk9\") pod \"network-check-target-5d9sj\" (UID: \"f009ab3e-edd9-4bf3-877e-37115c04d13f\") " pod="openshift-network-diagnostics/network-check-target-5d9sj"
Jul 23 23:13:32.838691 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:32.838610    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 23:13:32.838691 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:32.838635    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 23:13:32.838691 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:32.838646    2578 projected.go:196] Error preparing data for projected volume kube-api-access-k5bk9 for pod openshift-network-diagnostics/network-check-target-5d9sj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 23:13:32.838936 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:32.838716    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f009ab3e-edd9-4bf3-877e-37115c04d13f-kube-api-access-k5bk9 podName:f009ab3e-edd9-4bf3-877e-37115c04d13f nodeName:}" failed. No retries permitted until 2026-07-23 23:13:33.838698533 +0000 UTC m=+3.208224171 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-k5bk9" (UniqueName: "kubernetes.io/projected/f009ab3e-edd9-4bf3-877e-37115c04d13f-kube-api-access-k5bk9") pod "network-check-target-5d9sj" (UID: "f009ab3e-edd9-4bf3-877e-37115c04d13f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 23:13:33.082488 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:33.082283    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 23:08:33 +0000 UTC" deadline="2028-04-27 08:54:20.00307785 +0000 UTC"
Jul 23 23:13:33.082488 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:33.082348    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15441h40m46.920734488s"
Jul 23 23:13:33.186867 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:33.186838    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 23:13:33.236551 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:33.236512    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5d9sj"
Jul 23 23:13:33.236773 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:33.236646    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5d9sj" podUID="f009ab3e-edd9-4bf3-877e-37115c04d13f"
Jul 23 23:13:33.246619 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:33.246551    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzjsz" event={"ID":"61f6356e-0fdc-40c3-be10-216d774ff70b","Type":"ContainerStarted","Data":"4d1a6c2c3028bf048bd743e33e25060773f6645f4a2ed7d374333b7a36344bdc"}
Jul 23 23:13:33.258020 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:33.257853    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5" event={"ID":"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b","Type":"ContainerStarted","Data":"3b37a775521bd1b3ffa1e86e0d7e8e040fb789d9fd51db600a46f2e66e7597b5"}
Jul 23 23:13:33.261392 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:33.261106    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-v6zh5" event={"ID":"737833d3-8a38-4c00-8558-a7bccdc01903","Type":"ContainerStarted","Data":"3bfb65d4c9004ccff559ebd75647ac3692665cd0ad1da8547d7c99cabd2cf120"}
Jul 23 23:13:33.273796 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:33.273719    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rw58q" event={"ID":"18ba94f3-1bd7-41f4-a603-0a2654475a66","Type":"ContainerStarted","Data":"84e41283ab378212df5f4e09e06538b7d46a010fc2fe18eaf87d68ebd1e19936"}
Jul 23 23:13:33.280431 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:33.280389    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sn5k2" event={"ID":"6c30a49c-1943-4650-83f0-adaa3045404e","Type":"ContainerStarted","Data":"571e1f53950e26bd75822d65c747ba336d2c86247c1d38e93b6324c533217f0d"}
Jul 23 23:13:33.285147 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:33.285104    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9jn86" event={"ID":"960f9613-6e9d-4172-a662-b20bc07d8705","Type":"ContainerStarted","Data":"123e646f7db0d6df8598328afd5f1bfb411dfdef62eeae6a3dc37ca0bcf5b9db"}
Jul 23 23:13:33.294302 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:33.294259    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-f66wp" event={"ID":"59b943fa-86eb-453f-8e2f-63c64ce435ce","Type":"ContainerStarted","Data":"9fef8f02f7a3cb82f447df5eb2509bc7e156aa1e99c2e9701e6df89112c80900"}
Jul 23 23:13:33.307019 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:33.306972    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ljq8l" event={"ID":"d2943613-62a6-476e-bec2-12fb3c167e01","Type":"ContainerStarted","Data":"7c9a3a5b71f7bf8fdf2f9c67e2765fbf721dc5ba936c04dddeeff108e875bb6f"}
Jul 23 23:13:33.316897 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:33.316855    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-29zxk" event={"ID":"d1632b98-84f8-4658-af28-6521dd300a59","Type":"ContainerStarted","Data":"1b18034b76203bbd37a6a8b99c865186f68c45bc09dadb2b77e62af1b80883cb"}
Jul 23 23:13:33.321768 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:33.321716    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-165.ec2.internal" event={"ID":"f2a7eb7430412c2da89feeebacf09842","Type":"ContainerStarted","Data":"052cd3055f9c3f90da843d8ef448518d222e0bea8da21139dc0048a8ca59d20d"}
Jul 23 23:13:33.336631 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:33.336535    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-165.ec2.internal" event={"ID":"ef9bd6a40b72bafbfb43200d1ac0a01d","Type":"ContainerStarted","Data":"9dd22b22c4fcec23b58031406e8b6fce47c78a00aaf3a24ebfea07c09837b3ef"}
Jul 23 23:13:33.747202 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:33.747102    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dd112f5-f0cc-4e72-917c-33e4abca79cb-metrics-certs\") pod \"network-metrics-daemon-bbv68\" (UID: \"8dd112f5-f0cc-4e72-917c-33e4abca79cb\") " pod="openshift-multus/network-metrics-daemon-bbv68"
Jul 23 23:13:33.747202 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:33.747169    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b4d53a59-83bb-4725-841c-9f5b55f85ca4-original-pull-secret\") pod \"global-pull-secret-syncer-z8d42\" (UID: \"b4d53a59-83bb-4725-841c-9f5b55f85ca4\") " pod="kube-system/global-pull-secret-syncer-z8d42"
Jul 23 23:13:33.747461 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:33.747373    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 23:13:33.747461 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:33.747449    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4d53a59-83bb-4725-841c-9f5b55f85ca4-original-pull-secret podName:b4d53a59-83bb-4725-841c-9f5b55f85ca4 nodeName:}" failed. No retries permitted until 2026-07-23 23:13:35.747427961 +0000 UTC m=+5.116953603 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b4d53a59-83bb-4725-841c-9f5b55f85ca4-original-pull-secret") pod "global-pull-secret-syncer-z8d42" (UID: "b4d53a59-83bb-4725-841c-9f5b55f85ca4") : object "kube-system"/"original-pull-secret" not registered
Jul 23 23:13:33.747973 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:33.747948    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 23:13:33.748076 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:33.748019    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dd112f5-f0cc-4e72-917c-33e4abca79cb-metrics-certs podName:8dd112f5-f0cc-4e72-917c-33e4abca79cb nodeName:}" failed. No retries permitted until 2026-07-23 23:13:35.748000255 +0000 UTC m=+5.117525894 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8dd112f5-f0cc-4e72-917c-33e4abca79cb-metrics-certs") pod "network-metrics-daemon-bbv68" (UID: "8dd112f5-f0cc-4e72-917c-33e4abca79cb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 23:13:33.848238 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:33.848189    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5bk9\" (UniqueName: \"kubernetes.io/projected/f009ab3e-edd9-4bf3-877e-37115c04d13f-kube-api-access-k5bk9\") pod \"network-check-target-5d9sj\" (UID: \"f009ab3e-edd9-4bf3-877e-37115c04d13f\") " pod="openshift-network-diagnostics/network-check-target-5d9sj"
Jul 23 23:13:33.848452 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:33.848434    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 23:13:33.848523 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:33.848461    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 23:13:33.848523 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:33.848475    2578 projected.go:196] Error preparing data for projected volume kube-api-access-k5bk9 for pod openshift-network-diagnostics/network-check-target-5d9sj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 23:13:33.848656 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:33.848537    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f009ab3e-edd9-4bf3-877e-37115c04d13f-kube-api-access-k5bk9 podName:f009ab3e-edd9-4bf3-877e-37115c04d13f nodeName:}" failed. No retries permitted until 2026-07-23 23:13:35.848517837 +0000 UTC m=+5.218043498 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-k5bk9" (UniqueName: "kubernetes.io/projected/f009ab3e-edd9-4bf3-877e-37115c04d13f-kube-api-access-k5bk9") pod "network-check-target-5d9sj" (UID: "f009ab3e-edd9-4bf3-877e-37115c04d13f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 23:13:34.083407 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:34.083214    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 23:08:33 +0000 UTC" deadline="2028-02-28 00:48:17.170182454 +0000 UTC"
Jul 23 23:13:34.083407 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:34.083285    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14017h34m43.086903429s"
Jul 23 23:13:34.231096 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:34.231042    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z8d42"
Jul 23 23:13:34.231280 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:34.231251    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z8d42" podUID="b4d53a59-83bb-4725-841c-9f5b55f85ca4"
Jul 23 23:13:34.231766 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:34.231701    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bbv68"
Jul 23 23:13:34.231925 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:34.231873    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bbv68" podUID="8dd112f5-f0cc-4e72-917c-33e4abca79cb"
Jul 23 23:13:34.319153 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:34.319121    2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 23:13:35.231687 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:35.231110    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5d9sj"
Jul 23 23:13:35.231687 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:35.231261    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5d9sj" podUID="f009ab3e-edd9-4bf3-877e-37115c04d13f"
Jul 23 23:13:35.773190 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:35.773150    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dd112f5-f0cc-4e72-917c-33e4abca79cb-metrics-certs\") pod \"network-metrics-daemon-bbv68\" (UID: \"8dd112f5-f0cc-4e72-917c-33e4abca79cb\") " pod="openshift-multus/network-metrics-daemon-bbv68"
Jul 23 23:13:35.773402 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:35.773220    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b4d53a59-83bb-4725-841c-9f5b55f85ca4-original-pull-secret\") pod \"global-pull-secret-syncer-z8d42\" (UID: \"b4d53a59-83bb-4725-841c-9f5b55f85ca4\") " pod="kube-system/global-pull-secret-syncer-z8d42"
Jul 23 23:13:35.773402 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:35.773398    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 23:13:35.773530 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:35.773478    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4d53a59-83bb-4725-841c-9f5b55f85ca4-original-pull-secret podName:b4d53a59-83bb-4725-841c-9f5b55f85ca4 nodeName:}" failed. No retries permitted until 2026-07-23 23:13:39.77344626 +0000 UTC m=+9.142971903 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b4d53a59-83bb-4725-841c-9f5b55f85ca4-original-pull-secret") pod "global-pull-secret-syncer-z8d42" (UID: "b4d53a59-83bb-4725-841c-9f5b55f85ca4") : object "kube-system"/"original-pull-secret" not registered
Jul 23 23:13:35.773956 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:35.773933    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 23:13:35.774062 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:35.774005    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dd112f5-f0cc-4e72-917c-33e4abca79cb-metrics-certs podName:8dd112f5-f0cc-4e72-917c-33e4abca79cb nodeName:}" failed. No retries permitted until 2026-07-23 23:13:39.773986691 +0000 UTC m=+9.143512337 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8dd112f5-f0cc-4e72-917c-33e4abca79cb-metrics-certs") pod "network-metrics-daemon-bbv68" (UID: "8dd112f5-f0cc-4e72-917c-33e4abca79cb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 23:13:35.874494 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:35.874449    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5bk9\" (UniqueName: \"kubernetes.io/projected/f009ab3e-edd9-4bf3-877e-37115c04d13f-kube-api-access-k5bk9\") pod \"network-check-target-5d9sj\" (UID: \"f009ab3e-edd9-4bf3-877e-37115c04d13f\") " pod="openshift-network-diagnostics/network-check-target-5d9sj"
Jul 23 23:13:35.879051 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:35.875736    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 23:13:35.879051 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:35.875771    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 23:13:35.879051 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:35.875796    2578 projected.go:196] Error preparing data for projected volume kube-api-access-k5bk9 for pod openshift-network-diagnostics/network-check-target-5d9sj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 23:13:35.879051 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:35.875872    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f009ab3e-edd9-4bf3-877e-37115c04d13f-kube-api-access-k5bk9 podName:f009ab3e-edd9-4bf3-877e-37115c04d13f nodeName:}" failed. No retries permitted until 2026-07-23 23:13:39.875841937 +0000 UTC m=+9.245367580 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-k5bk9" (UniqueName: "kubernetes.io/projected/f009ab3e-edd9-4bf3-877e-37115c04d13f-kube-api-access-k5bk9") pod "network-check-target-5d9sj" (UID: "f009ab3e-edd9-4bf3-877e-37115c04d13f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 23:13:36.231240 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:36.231149    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bbv68"
Jul 23 23:13:36.231422 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:36.231309    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bbv68" podUID="8dd112f5-f0cc-4e72-917c-33e4abca79cb"
Jul 23 23:13:36.231805 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:36.231781    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z8d42"
Jul 23 23:13:36.232238 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:36.231920    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z8d42" podUID="b4d53a59-83bb-4725-841c-9f5b55f85ca4"
Jul 23 23:13:37.236985 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:37.236459    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5d9sj"
Jul 23 23:13:37.236985 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:37.236590    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5d9sj" podUID="f009ab3e-edd9-4bf3-877e-37115c04d13f"
Jul 23 23:13:38.230234 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:38.230199    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z8d42"
Jul 23 23:13:38.230408 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:38.230328    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z8d42" podUID="b4d53a59-83bb-4725-841c-9f5b55f85ca4"
Jul 23 23:13:38.230949 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:38.230776    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bbv68"
Jul 23 23:13:38.230949 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:38.230897    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bbv68" podUID="8dd112f5-f0cc-4e72-917c-33e4abca79cb"
Jul 23 23:13:39.233143 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:39.233108    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5d9sj"
Jul 23 23:13:39.233656 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:39.233254    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5d9sj" podUID="f009ab3e-edd9-4bf3-877e-37115c04d13f"
Jul 23 23:13:39.809978 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:39.809942    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dd112f5-f0cc-4e72-917c-33e4abca79cb-metrics-certs\") pod \"network-metrics-daemon-bbv68\" (UID: \"8dd112f5-f0cc-4e72-917c-33e4abca79cb\") " pod="openshift-multus/network-metrics-daemon-bbv68"
Jul 23 23:13:39.810175 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:39.810005    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b4d53a59-83bb-4725-841c-9f5b55f85ca4-original-pull-secret\") pod \"global-pull-secret-syncer-z8d42\" (UID: \"b4d53a59-83bb-4725-841c-9f5b55f85ca4\") " pod="kube-system/global-pull-secret-syncer-z8d42"
Jul 23 23:13:39.810175 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:39.810071    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 23:13:39.810175 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:39.810149    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dd112f5-f0cc-4e72-917c-33e4abca79cb-metrics-certs podName:8dd112f5-f0cc-4e72-917c-33e4abca79cb nodeName:}" failed. No retries permitted until 2026-07-23 23:13:47.810130867 +0000 UTC m=+17.179656524 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8dd112f5-f0cc-4e72-917c-33e4abca79cb-metrics-certs") pod "network-metrics-daemon-bbv68" (UID: "8dd112f5-f0cc-4e72-917c-33e4abca79cb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 23:13:39.810175 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:39.810165    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 23:13:39.810428 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:39.810234    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4d53a59-83bb-4725-841c-9f5b55f85ca4-original-pull-secret podName:b4d53a59-83bb-4725-841c-9f5b55f85ca4 nodeName:}" failed. No retries permitted until 2026-07-23 23:13:47.810220631 +0000 UTC m=+17.179746275 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b4d53a59-83bb-4725-841c-9f5b55f85ca4-original-pull-secret") pod "global-pull-secret-syncer-z8d42" (UID: "b4d53a59-83bb-4725-841c-9f5b55f85ca4") : object "kube-system"/"original-pull-secret" not registered
Jul 23 23:13:39.910938 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:39.910898    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5bk9\" (UniqueName: \"kubernetes.io/projected/f009ab3e-edd9-4bf3-877e-37115c04d13f-kube-api-access-k5bk9\") pod \"network-check-target-5d9sj\" (UID: \"f009ab3e-edd9-4bf3-877e-37115c04d13f\") " pod="openshift-network-diagnostics/network-check-target-5d9sj"
Jul 23 23:13:39.911136 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:39.911106    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 23:13:39.911136 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:39.911128    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 23:13:39.911236 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:39.911142    2578 projected.go:196] Error preparing data for projected volume kube-api-access-k5bk9 for pod openshift-network-diagnostics/network-check-target-5d9sj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 23:13:39.911236 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:39.911207    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f009ab3e-edd9-4bf3-877e-37115c04d13f-kube-api-access-k5bk9 podName:f009ab3e-edd9-4bf3-877e-37115c04d13f nodeName:}" failed. No retries permitted until 2026-07-23 23:13:47.911186038 +0000 UTC m=+17.280711682 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-k5bk9" (UniqueName: "kubernetes.io/projected/f009ab3e-edd9-4bf3-877e-37115c04d13f-kube-api-access-k5bk9") pod "network-check-target-5d9sj" (UID: "f009ab3e-edd9-4bf3-877e-37115c04d13f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 23:13:40.231360 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:40.231265    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z8d42"
Jul 23 23:13:40.231360 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:40.231324    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bbv68"
Jul 23 23:13:40.231598 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:40.231469    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z8d42" podUID="b4d53a59-83bb-4725-841c-9f5b55f85ca4"
Jul 23 23:13:40.231598 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:40.231580    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bbv68" podUID="8dd112f5-f0cc-4e72-917c-33e4abca79cb"
Jul 23 23:13:41.232046 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:41.232000    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5d9sj"
Jul 23 23:13:41.232546 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:41.232153    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5d9sj" podUID="f009ab3e-edd9-4bf3-877e-37115c04d13f"
Jul 23 23:13:42.230809 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:42.230772    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z8d42"
Jul 23 23:13:42.231016 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:42.230905    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z8d42" podUID="b4d53a59-83bb-4725-841c-9f5b55f85ca4"
Jul 23 23:13:42.231144 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:42.230772    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bbv68"
Jul 23 23:13:42.231278 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:42.231245    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bbv68" podUID="8dd112f5-f0cc-4e72-917c-33e4abca79cb"
Jul 23 23:13:43.230924 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:43.230886    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5d9sj"
Jul 23 23:13:43.231353 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:43.231046    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5d9sj" podUID="f009ab3e-edd9-4bf3-877e-37115c04d13f"
Jul 23 23:13:44.231036 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:44.230998    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z8d42"
Jul 23 23:13:44.231430 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:44.231000    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bbv68"
Jul 23 23:13:44.231430 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:44.231170    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z8d42" podUID="b4d53a59-83bb-4725-841c-9f5b55f85ca4"
Jul 23 23:13:44.231430 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:44.231305    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bbv68" podUID="8dd112f5-f0cc-4e72-917c-33e4abca79cb"
Jul 23 23:13:45.230695 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:45.230636    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5d9sj"
Jul 23 23:13:45.230876 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:45.230789    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5d9sj" podUID="f009ab3e-edd9-4bf3-877e-37115c04d13f"
Jul 23 23:13:46.230878 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:46.230841    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z8d42"
Jul 23 23:13:46.230878 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:46.230878    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bbv68"
Jul 23 23:13:46.231379 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:46.230984    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z8d42" podUID="b4d53a59-83bb-4725-841c-9f5b55f85ca4"
Jul 23 23:13:46.231379 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:46.231143    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bbv68" podUID="8dd112f5-f0cc-4e72-917c-33e4abca79cb"
Jul 23 23:13:47.231004 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:47.230968    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5d9sj"
Jul 23 23:13:47.231414 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:47.231104    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5d9sj" podUID="f009ab3e-edd9-4bf3-877e-37115c04d13f"
Jul 23 23:13:47.884089 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:47.884051    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dd112f5-f0cc-4e72-917c-33e4abca79cb-metrics-certs\") pod \"network-metrics-daemon-bbv68\" (UID: \"8dd112f5-f0cc-4e72-917c-33e4abca79cb\") " pod="openshift-multus/network-metrics-daemon-bbv68"
Jul 23 23:13:47.884089 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:47.884107    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b4d53a59-83bb-4725-841c-9f5b55f85ca4-original-pull-secret\") pod \"global-pull-secret-syncer-z8d42\" (UID: \"b4d53a59-83bb-4725-841c-9f5b55f85ca4\") " pod="kube-system/global-pull-secret-syncer-z8d42"
Jul 23 23:13:47.884329 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:47.884244    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 23:13:47.884329 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:47.884308    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4d53a59-83bb-4725-841c-9f5b55f85ca4-original-pull-secret podName:b4d53a59-83bb-4725-841c-9f5b55f85ca4 nodeName:}" failed. No retries permitted until 2026-07-23 23:14:03.884291479 +0000 UTC m=+33.253817118 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b4d53a59-83bb-4725-841c-9f5b55f85ca4-original-pull-secret") pod "global-pull-secret-syncer-z8d42" (UID: "b4d53a59-83bb-4725-841c-9f5b55f85ca4") : object "kube-system"/"original-pull-secret" not registered
Jul 23 23:13:47.884783 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:47.884757    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 23:13:47.884892 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:47.884816    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dd112f5-f0cc-4e72-917c-33e4abca79cb-metrics-certs podName:8dd112f5-f0cc-4e72-917c-33e4abca79cb nodeName:}" failed. No retries permitted until 2026-07-23 23:14:03.884801998 +0000 UTC m=+33.254327635 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8dd112f5-f0cc-4e72-917c-33e4abca79cb-metrics-certs") pod "network-metrics-daemon-bbv68" (UID: "8dd112f5-f0cc-4e72-917c-33e4abca79cb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 23:13:47.985229 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:47.985190    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5bk9\" (UniqueName: \"kubernetes.io/projected/f009ab3e-edd9-4bf3-877e-37115c04d13f-kube-api-access-k5bk9\") pod \"network-check-target-5d9sj\" (UID: \"f009ab3e-edd9-4bf3-877e-37115c04d13f\") " pod="openshift-network-diagnostics/network-check-target-5d9sj"
Jul 23 23:13:47.985437 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:47.985417    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 23:13:47.985503 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:47.985446    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 23:13:47.985503 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:47.985461    2578 projected.go:196] Error preparing data for projected volume kube-api-access-k5bk9 for pod openshift-network-diagnostics/network-check-target-5d9sj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 23:13:47.985592 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:47.985518    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f009ab3e-edd9-4bf3-877e-37115c04d13f-kube-api-access-k5bk9 podName:f009ab3e-edd9-4bf3-877e-37115c04d13f nodeName:}" failed. No retries permitted until 2026-07-23 23:14:03.985501232 +0000 UTC m=+33.355026885 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-k5bk9" (UniqueName: "kubernetes.io/projected/f009ab3e-edd9-4bf3-877e-37115c04d13f-kube-api-access-k5bk9") pod "network-check-target-5d9sj" (UID: "f009ab3e-edd9-4bf3-877e-37115c04d13f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 23:13:48.231134 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:48.231049    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z8d42"
Jul 23 23:13:48.231134 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:48.231065    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bbv68"
Jul 23 23:13:48.231606 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:48.231189    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z8d42" podUID="b4d53a59-83bb-4725-841c-9f5b55f85ca4"
Jul 23 23:13:48.231606 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:48.231588    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bbv68" podUID="8dd112f5-f0cc-4e72-917c-33e4abca79cb"
Jul 23 23:13:49.230564 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:49.230379    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5d9sj"
Jul 23 23:13:49.230771 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:49.230681    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5d9sj" podUID="f009ab3e-edd9-4bf3-877e-37115c04d13f"
Jul 23 23:13:50.231151 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:50.231111    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bbv68"
Jul 23 23:13:50.231578 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:50.231155    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z8d42"
Jul 23 23:13:50.231578 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:50.231267    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bbv68" podUID="8dd112f5-f0cc-4e72-917c-33e4abca79cb"
Jul 23 23:13:50.231578 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:50.231397    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z8d42" podUID="b4d53a59-83bb-4725-841c-9f5b55f85ca4"
Jul 23 23:13:51.233077 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:51.233047    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5d9sj"
Jul 23 23:13:51.233563 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:51.233147    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5d9sj" podUID="f009ab3e-edd9-4bf3-877e-37115c04d13f"
Jul 23 23:13:52.230839 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:52.230801    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bbv68"
Jul 23 23:13:52.231011 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:52.230860    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z8d42"
Jul 23 23:13:52.231011 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:52.230954    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bbv68" podUID="8dd112f5-f0cc-4e72-917c-33e4abca79cb"
Jul 23 23:13:52.231138 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:52.231086    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z8d42" podUID="b4d53a59-83bb-4725-841c-9f5b55f85ca4"
Jul 23 23:13:53.230517 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:53.230481    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5d9sj"
Jul 23 23:13:53.231147 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:53.230592    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5d9sj" podUID="f009ab3e-edd9-4bf3-877e-37115c04d13f"
Jul 23 23:13:54.230729 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:54.230525    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z8d42"
Jul 23 23:13:54.231309 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:54.230526    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bbv68"
Jul 23 23:13:54.231309 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:54.230825    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z8d42" podUID="b4d53a59-83bb-4725-841c-9f5b55f85ca4"
Jul 23 23:13:54.231309 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:54.230909    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bbv68" podUID="8dd112f5-f0cc-4e72-917c-33e4abca79cb"
Jul 23 23:13:54.406274 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:54.406238    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-29zxk" event={"ID":"d1632b98-84f8-4658-af28-6521dd300a59","Type":"ContainerStarted","Data":"4ffcd1148b5ff0b9ab98a730993bee4d29f342ee6ca6706e8622f738e0f4ba0e"}
Jul 23 23:13:54.407698 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:54.407672    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-165.ec2.internal" event={"ID":"f2a7eb7430412c2da89feeebacf09842","Type":"ContainerStarted","Data":"e4e92443168038b2d0a840090ff1c629e7ba013507a8295e8bb904489d24a2ce"}
Jul 23 23:13:54.408910 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:54.408887    2578 generic.go:356] "Generic (PLEG): container finished" podID="ef9bd6a40b72bafbfb43200d1ac0a01d" containerID="eeb3e8032b4640b3bfdcc40031d4f80ef5f843df792f7eb4124ac3298dcde7e1" exitCode=0
Jul 23 23:13:54.409006 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:54.408959    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-165.ec2.internal" event={"ID":"ef9bd6a40b72bafbfb43200d1ac0a01d","Type":"ContainerDied","Data":"eeb3e8032b4640b3bfdcc40031d4f80ef5f843df792f7eb4124ac3298dcde7e1"}
Jul 23 23:13:54.410261 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:54.410242    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzjsz" event={"ID":"61f6356e-0fdc-40c3-be10-216d774ff70b","Type":"ContainerStarted","Data":"2824aadc0f1c8862dfedb81b8ca1282d7d5e8a0e7dbcea5292620d87c39ed6ae"}
Jul 23 23:13:54.413047 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:54.413022    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5" event={"ID":"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b","Type":"ContainerStarted","Data":"31c5425ec444209ab1bd09f5b810a6118da0c2f5a25a2bce3ef2198650bc1368"}
Jul 23 23:13:54.413138 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:54.413073    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5" event={"ID":"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b","Type":"ContainerStarted","Data":"2fd0f412662f82269e62a94a9a0522aa55ec6b23a5ed831dc39334ae21617871"}
Jul 23 23:13:54.413138 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:54.413103    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5" event={"ID":"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b","Type":"ContainerStarted","Data":"49affe43c34849c3ff3141a708e0e31d2a72ffa481e90ad0002c3eb8146c103d"}
Jul 23 23:13:54.413138 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:54.413130    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5" event={"ID":"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b","Type":"ContainerStarted","Data":"74613d9726e90f54d65c98b16a040e50277d6d3a8776ea8bef52db88e13efa2f"}
Jul 23 23:13:54.413283 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:54.413161    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5" event={"ID":"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b","Type":"ContainerStarted","Data":"06703021c0f18a43d9c35e76691840e17342c59b2029bab0c4b222e57e68190d"}
Jul 23 23:13:54.413283 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:54.413193    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5" event={"ID":"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b","Type":"ContainerStarted","Data":"539efecf2c9c5524bd90174c900202639d1ddfed44e40af35c2bdfc16b36c41c"}
Jul 23 23:13:54.417028 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:54.417000    2578 generic.go:356] "Generic (PLEG): container finished" podID="18ba94f3-1bd7-41f4-a603-0a2654475a66" containerID="b41d77b6d0486c83de8d4fc5c5bf748451eea74feb83d436ae707b21b988155e" exitCode=0
Jul 23 23:13:54.417136 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:54.417029    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rw58q" event={"ID":"18ba94f3-1bd7-41f4-a603-0a2654475a66","Type":"ContainerDied","Data":"b41d77b6d0486c83de8d4fc5c5bf748451eea74feb83d436ae707b21b988155e"}
Jul 23 23:13:54.418503 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:54.418433    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sn5k2" event={"ID":"6c30a49c-1943-4650-83f0-adaa3045404e","Type":"ContainerStarted","Data":"2a284cf42166df4a9658f03a235e453a0bf63facbcb2302eb727bbbf6efbfc3b"}
Jul 23 23:13:54.419771 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:54.419750    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9jn86" event={"ID":"960f9613-6e9d-4172-a662-b20bc07d8705","Type":"ContainerStarted","Data":"4298ef002776e1038cb738f6b388d960d7063dabf4f67dbd34240a0d6dbfadfa"}
Jul 23 23:13:54.421154 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:54.421135    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-f66wp" event={"ID":"59b943fa-86eb-453f-8e2f-63c64ce435ce","Type":"ContainerStarted","Data":"7c40138e17322aadb8237130cdb9949ac1f6ec49b6f3cee5255afbf19fc28977"}
Jul 23 23:13:54.422438 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:54.422419    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ljq8l" event={"ID":"d2943613-62a6-476e-bec2-12fb3c167e01","Type":"ContainerStarted","Data":"601781e00af2f49a1b64658d552680a287980774949c7e842cbbd3e200a82590"}
Jul 23 23:13:54.423772 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:54.423737    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-29zxk" podStartSLOduration=1.201741557 podStartE2EDuration="22.423722641s" totalImagesPullingTime="21.221981084s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 23:13:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:13:54.423574316 +0000 UTC m=+23.793099975" watchObservedRunningTime="2026-07-23 23:13:54.423722641 +0000 UTC m=+23.793248301"
Jul 23 23:13:54.440162 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:54.440100    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-165.ec2.internal" podStartSLOduration=22.440084814 podStartE2EDuration="22.440084814s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 23:13:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:13:54.43989473 +0000 UTC m=+23.809420389" watchObservedRunningTime="2026-07-23 23:13:54.440084814 +0000 UTC m=+23.809610469"
Jul 23 23:13:54.458861 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:54.458813    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-9jn86" podStartSLOduration=1.269763626 podStartE2EDuration="22.458795733s" totalImagesPullingTime="21.189032107s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:13:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:13:54.458440866 +0000 UTC m=+23.827966526" watchObservedRunningTime="2026-07-23 23:13:54.458795733 +0000 UTC m=+23.828321391"
Jul 23 23:13:54.497619 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:54.497563    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-ljq8l" podStartSLOduration=9.401559276 podStartE2EDuration="22.497547111s" totalImagesPullingTime="13.095987835s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 23:13:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:13:54.496941732 +0000 UTC m=+23.866467390" watchObservedRunningTime="2026-07-23 23:13:54.497547111 +0000 UTC m=+23.867072770"
Jul 23 23:13:54.514536 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:54.514472    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-f66wp" podStartSLOduration=1.233706739 podStartE2EDuration="22.514456504s" totalImagesPullingTime="21.280749765s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 23:13:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:13:54.513891707 +0000 UTC m=+23.883417366" watchObservedRunningTime="2026-07-23 23:13:54.514456504 +0000 UTC m=+23.883982210"
Jul 23 23:13:54.528467 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:54.528414    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-sn5k2" podStartSLOduration=1.36340568 podStartE2EDuration="22.528393031s" totalImagesPullingTime="21.164987351s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 23:13:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:13:54.527735527 +0000 UTC m=+23.897261189" watchObservedRunningTime="2026-07-23 23:13:54.528393031 +0000 UTC m=+23.897918691"
Jul 23 23:13:55.231309 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:55.231092    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5d9sj"
Jul 23 23:13:55.231924 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:55.231398    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5d9sj" podUID="f009ab3e-edd9-4bf3-877e-37115c04d13f"
Jul 23 23:13:55.413037 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:55.412999    2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 23 23:13:55.426265 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:55.426236    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzjsz" event={"ID":"61f6356e-0fdc-40c3-be10-216d774ff70b","Type":"ContainerStarted","Data":"4a400429dc08af9903766cfe0a628a3b5f627b39205eae8bef35c62be0470eb2"}
Jul 23 23:13:55.427519 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:55.427491    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-v6zh5" event={"ID":"737833d3-8a38-4c00-8558-a7bccdc01903","Type":"ContainerStarted","Data":"b125da956b9813415556ba6e21eef5c8e5652eb36b518f52e8551866e5d9235e"}
Jul 23 23:13:55.456725 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:55.456671    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-v6zh5" podStartSLOduration=2.336275332 podStartE2EDuration="23.456637455s" totalImagesPullingTime="21.120362123s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 23:13:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:13:55.456195365 +0000 UTC m=+24.825721021" watchObservedRunningTime="2026-07-23 23:13:55.456637455 +0000 UTC m=+24.826163114"
Jul 23 23:13:56.093635 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:56.093506    2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-23T23:13:55.413027978Z","UUID":"6cc870bd-fbfd-4169-a0f9-2cdb7a32bde2","Handler":null,"Name":"","Endpoint":""}
Jul 23 23:13:56.095348 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:56.095324    2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 23 23:13:56.095498 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:56.095370    2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 23 23:13:56.167237 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:56.167204    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-9jn86"
Jul 23 23:13:56.168063 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:56.168042    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-9jn86"
Jul 23 23:13:56.230876 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:56.230843    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bbv68"
Jul 23 23:13:56.230876 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:56.230864    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z8d42"
Jul 23 23:13:56.231102 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:56.230984    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bbv68" podUID="8dd112f5-f0cc-4e72-917c-33e4abca79cb"
Jul 23 23:13:56.231155 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:56.231120    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z8d42" podUID="b4d53a59-83bb-4725-841c-9f5b55f85ca4"
Jul 23 23:13:56.432668 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:56.432566    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-165.ec2.internal" event={"ID":"ef9bd6a40b72bafbfb43200d1ac0a01d","Type":"ContainerStarted","Data":"04c5b1075039ce22a07cc8493b1c37afb4d15a236fa3e7950c87a5251c7f9f5d"}
Jul 23 23:13:56.433107 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:56.432987    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-9jn86"
Jul 23 23:13:56.433537 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:56.433513    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9jn86"
Jul 23 23:13:56.454700 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:56.454602    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-165.ec2.internal" podStartSLOduration=24.454588179 podStartE2EDuration="24.454588179s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 23:13:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:13:56.447745133 +0000 UTC m=+25.817270793" watchObservedRunningTime="2026-07-23 23:13:56.454588179 +0000 UTC m=+25.824113838"
Jul 23 23:13:57.231094 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:57.231058    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5d9sj"
Jul 23 23:13:57.231295 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:57.231184    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5d9sj" podUID="f009ab3e-edd9-4bf3-877e-37115c04d13f"
Jul 23 23:13:57.437455 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:57.437418    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzjsz" event={"ID":"61f6356e-0fdc-40c3-be10-216d774ff70b","Type":"ContainerStarted","Data":"76bc73ae5cd6c762d0cac0037dfac870b6ee64eddeab2665d3a225733bb7aad0"}
Jul 23 23:13:57.441459 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:57.441366    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5" event={"ID":"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b","Type":"ContainerStarted","Data":"fa103722f0883d9ced810d80171eb0312dd4d979e96822dffe2970890e97167c"}
Jul 23 23:13:57.456545 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:57.456486    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nzjsz" podStartSLOduration=1.926346178 podStartE2EDuration="25.456462367s" totalImagesPullingTime="23.530116189s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 23:13:32 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:13:57.456296293 +0000 UTC m=+26.825821951" watchObservedRunningTime="2026-07-23 23:13:57.456462367 +0000 UTC m=+26.825988027"
Jul 23 23:13:58.231292 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:58.231246    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bbv68"
Jul 23 23:13:58.231481 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:58.231297    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z8d42"
Jul 23 23:13:58.231481 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:58.231383    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bbv68" podUID="8dd112f5-f0cc-4e72-917c-33e4abca79cb"
Jul 23 23:13:58.231592 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:58.231515    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z8d42" podUID="b4d53a59-83bb-4725-841c-9f5b55f85ca4"
Jul 23 23:13:59.231192 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:13:59.231120    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5d9sj"
Jul 23 23:13:59.231604 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:13:59.231251    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5d9sj" podUID="f009ab3e-edd9-4bf3-877e-37115c04d13f"
Jul 23 23:14:00.231358 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:00.231096    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bbv68"
Jul 23 23:14:00.232020 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:00.231135    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z8d42"
Jul 23 23:14:00.232020 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:14:00.231403    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bbv68" podUID="8dd112f5-f0cc-4e72-917c-33e4abca79cb"
Jul 23 23:14:00.232020 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:14:00.231456    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z8d42" podUID="b4d53a59-83bb-4725-841c-9f5b55f85ca4"
Jul 23 23:14:00.451789 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:00.451757    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5" event={"ID":"ca6b871c-0d2a-44cb-832d-2fd4f7e6395b","Type":"ContainerStarted","Data":"8ee21bc0a761ac99122588699659c0137f7d4c7a0eef976f49f0069205075387"}
Jul 23 23:14:00.452008 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:00.451918    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:14:00.452072 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:00.452038    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:14:00.453724 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:00.453693    2578 generic.go:356] "Generic (PLEG): container finished" podID="18ba94f3-1bd7-41f4-a603-0a2654475a66" containerID="f162f67d1e77aeb716564c43520059343be88be3e699629612d91ed176412aa4" exitCode=0
Jul 23 23:14:00.453820 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:00.453748    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rw58q" event={"ID":"18ba94f3-1bd7-41f4-a603-0a2654475a66","Type":"ContainerDied","Data":"f162f67d1e77aeb716564c43520059343be88be3e699629612d91ed176412aa4"}
Jul 23 23:14:00.468735 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:00.468712    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:14:00.480465 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:00.480418    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5" podStartSLOduration=7.303213037 podStartE2EDuration="28.480402561s" totalImagesPullingTime="21.177189524s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 23:13:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:14:00.480047859 +0000 UTC m=+29.849573518" watchObservedRunningTime="2026-07-23 23:14:00.480402561 +0000 UTC m=+29.849928220"
Jul 23 23:14:01.232154 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:01.232117    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5d9sj"
Jul 23 23:14:01.232725 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:14:01.232238    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5d9sj" podUID="f009ab3e-edd9-4bf3-877e-37115c04d13f"
Jul 23 23:14:01.456306 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:01.456282    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:14:01.471913 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:01.471889    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:14:02.230965 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:02.230930    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bbv68"
Jul 23 23:14:02.231163 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:14:02.231065    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bbv68" podUID="8dd112f5-f0cc-4e72-917c-33e4abca79cb"
Jul 23 23:14:02.231163 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:02.231111    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z8d42"
Jul 23 23:14:02.231276 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:14:02.231196    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z8d42" podUID="b4d53a59-83bb-4725-841c-9f5b55f85ca4"
Jul 23 23:14:02.459826 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:02.459794    2578 generic.go:356] "Generic (PLEG): container finished" podID="18ba94f3-1bd7-41f4-a603-0a2654475a66" containerID="82c4fc8c1ed581efb28b261582e4ab75771932a668bef894b0d04942114019c1" exitCode=0
Jul 23 23:14:02.460210 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:02.459883    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rw58q" event={"ID":"18ba94f3-1bd7-41f4-a603-0a2654475a66","Type":"ContainerDied","Data":"82c4fc8c1ed581efb28b261582e4ab75771932a668bef894b0d04942114019c1"}
Jul 23 23:14:03.231401 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:03.231360    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5d9sj"
Jul 23 23:14:03.231592 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:14:03.231509    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5d9sj" podUID="f009ab3e-edd9-4bf3-877e-37115c04d13f"
Jul 23 23:14:03.903679 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:03.903486    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dd112f5-f0cc-4e72-917c-33e4abca79cb-metrics-certs\") pod \"network-metrics-daemon-bbv68\" (UID: \"8dd112f5-f0cc-4e72-917c-33e4abca79cb\") " pod="openshift-multus/network-metrics-daemon-bbv68"
Jul 23 23:14:03.903960 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:14:03.903682    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 23:14:03.903960 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:03.903700    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b4d53a59-83bb-4725-841c-9f5b55f85ca4-original-pull-secret\") pod \"global-pull-secret-syncer-z8d42\" (UID: \"b4d53a59-83bb-4725-841c-9f5b55f85ca4\") " pod="kube-system/global-pull-secret-syncer-z8d42"
Jul 23 23:14:03.903960 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:14:03.903762    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 23:14:03.903960 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:14:03.903780    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dd112f5-f0cc-4e72-917c-33e4abca79cb-metrics-certs podName:8dd112f5-f0cc-4e72-917c-33e4abca79cb nodeName:}" failed. No retries permitted until 2026-07-23 23:14:35.903760272 +0000 UTC m=+65.273285912 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8dd112f5-f0cc-4e72-917c-33e4abca79cb-metrics-certs") pod "network-metrics-daemon-bbv68" (UID: "8dd112f5-f0cc-4e72-917c-33e4abca79cb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 23:14:03.903960 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:14:03.903817    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4d53a59-83bb-4725-841c-9f5b55f85ca4-original-pull-secret podName:b4d53a59-83bb-4725-841c-9f5b55f85ca4 nodeName:}" failed. No retries permitted until 2026-07-23 23:14:35.903807227 +0000 UTC m=+65.273332867 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b4d53a59-83bb-4725-841c-9f5b55f85ca4-original-pull-secret") pod "global-pull-secret-syncer-z8d42" (UID: "b4d53a59-83bb-4725-841c-9f5b55f85ca4") : object "kube-system"/"original-pull-secret" not registered
Jul 23 23:14:04.006953 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:04.006922    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5bk9\" (UniqueName: \"kubernetes.io/projected/f009ab3e-edd9-4bf3-877e-37115c04d13f-kube-api-access-k5bk9\") pod \"network-check-target-5d9sj\" (UID: \"f009ab3e-edd9-4bf3-877e-37115c04d13f\") " pod="openshift-network-diagnostics/network-check-target-5d9sj"
Jul 23 23:14:04.007112 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:14:04.007042    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 23:14:04.007112 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:14:04.007056    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 23:14:04.007112 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:14:04.007064    2578 projected.go:196] Error preparing data for projected volume kube-api-access-k5bk9 for pod openshift-network-diagnostics/network-check-target-5d9sj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 23:14:04.007112 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:14:04.007113    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f009ab3e-edd9-4bf3-877e-37115c04d13f-kube-api-access-k5bk9 podName:f009ab3e-edd9-4bf3-877e-37115c04d13f nodeName:}" failed. No retries permitted until 2026-07-23 23:14:36.007099586 +0000 UTC m=+65.376625223 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-k5bk9" (UniqueName: "kubernetes.io/projected/f009ab3e-edd9-4bf3-877e-37115c04d13f-kube-api-access-k5bk9") pod "network-check-target-5d9sj" (UID: "f009ab3e-edd9-4bf3-877e-37115c04d13f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 23:14:04.231016 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:04.230922    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bbv68"
Jul 23 23:14:04.231185 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:04.230921    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z8d42"
Jul 23 23:14:04.231185 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:14:04.231034    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bbv68" podUID="8dd112f5-f0cc-4e72-917c-33e4abca79cb"
Jul 23 23:14:04.231185 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:14:04.231136    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z8d42" podUID="b4d53a59-83bb-4725-841c-9f5b55f85ca4"
Jul 23 23:14:04.467482 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:04.467445    2578 generic.go:356] "Generic (PLEG): container finished" podID="18ba94f3-1bd7-41f4-a603-0a2654475a66" containerID="1efbbcfbe00672817a82bee928cba6aa0870d567ad2b98e099eeebe34cce8136" exitCode=0
Jul 23 23:14:04.467695 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:04.467496    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rw58q" event={"ID":"18ba94f3-1bd7-41f4-a603-0a2654475a66","Type":"ContainerDied","Data":"1efbbcfbe00672817a82bee928cba6aa0870d567ad2b98e099eeebe34cce8136"}
Jul 23 23:14:05.230344 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:05.230314    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5d9sj"
Jul 23 23:14:05.230827 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:14:05.230478    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5d9sj" podUID="f009ab3e-edd9-4bf3-877e-37115c04d13f"
Jul 23 23:14:05.525059 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:05.524968    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bbv68"]
Jul 23 23:14:05.525205 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:05.525093    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-z8d42"]
Jul 23 23:14:05.525205 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:05.525190    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z8d42"
Jul 23 23:14:05.528059 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:14:05.525339    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z8d42" podUID="b4d53a59-83bb-4725-841c-9f5b55f85ca4"
Jul 23 23:14:05.528059 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:05.527570    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bbv68"
Jul 23 23:14:05.528059 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:14:05.527738    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bbv68" podUID="8dd112f5-f0cc-4e72-917c-33e4abca79cb"
Jul 23 23:14:05.530060 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:05.530021    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5d9sj"]
Jul 23 23:14:05.530209 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:05.530161    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5d9sj"
Jul 23 23:14:05.530341 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:14:05.530316    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5d9sj" podUID="f009ab3e-edd9-4bf3-877e-37115c04d13f"
Jul 23 23:14:07.230892 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:07.230854    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bbv68"
Jul 23 23:14:07.231322 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:07.230858    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5d9sj"
Jul 23 23:14:07.231322 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:07.230989    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z8d42"
Jul 23 23:14:07.231322 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:14:07.230986    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bbv68" podUID="8dd112f5-f0cc-4e72-917c-33e4abca79cb"
Jul 23 23:14:07.231322 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:14:07.231049    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5d9sj" podUID="f009ab3e-edd9-4bf3-877e-37115c04d13f"
Jul 23 23:14:07.231322 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:14:07.231123    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z8d42" podUID="b4d53a59-83bb-4725-841c-9f5b55f85ca4"
Jul 23 23:14:09.230296 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:09.230214    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5d9sj"
Jul 23 23:14:09.230296 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:09.230264    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z8d42"
Jul 23 23:14:09.230714 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:09.230329    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bbv68"
Jul 23 23:14:09.230714 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:14:09.230349    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5d9sj" podUID="f009ab3e-edd9-4bf3-877e-37115c04d13f"
Jul 23 23:14:09.230714 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:14:09.230407    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z8d42" podUID="b4d53a59-83bb-4725-841c-9f5b55f85ca4"
Jul 23 23:14:09.230714 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:14:09.230485    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bbv68" podUID="8dd112f5-f0cc-4e72-917c-33e4abca79cb"
Jul 23 23:14:09.482750 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:09.482466    2578 generic.go:356] "Generic (PLEG): container finished" podID="18ba94f3-1bd7-41f4-a603-0a2654475a66" containerID="a19f6d1d557d4495198a8c915f117e0e7439be535404992b388002b7087e2993" exitCode=0
Jul 23 23:14:09.482750 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:09.482545    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rw58q" event={"ID":"18ba94f3-1bd7-41f4-a603-0a2654475a66","Type":"ContainerDied","Data":"a19f6d1d557d4495198a8c915f117e0e7439be535404992b388002b7087e2993"}
Jul 23 23:14:10.406763 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.406692    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-165.ec2.internal" event="NodeReady"
Jul 23 23:14:10.407103 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.406836    2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 23 23:14:10.453521 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.453486    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9bpqj"]
Jul 23 23:14:10.455444 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.455424    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9bpqj"
Jul 23 23:14:10.458386 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.458361    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-m52bp\""
Jul 23 23:14:10.458522 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.458430    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 23 23:14:10.458780 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.458762    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 23 23:14:10.458946 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.458928    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-6pgdz"]
Jul 23 23:14:10.460616 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.460597    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6pgdz"
Jul 23 23:14:10.463642 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.463620    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ckkjh\""
Jul 23 23:14:10.463769 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.463730    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 23 23:14:10.463769 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.463764    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 23 23:14:10.463877 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.463858    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 23 23:14:10.463963 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.463949    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 23 23:14:10.469925 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.469891    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9bpqj"]
Jul 23 23:14:10.472847 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.472818    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6pgdz"]
Jul 23 23:14:10.476233 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.476207    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hsq95"]
Jul 23 23:14:10.478031 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.478014    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hsq95"
Jul 23 23:14:10.480857 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.480835    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 23 23:14:10.480985 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.480893    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 23 23:14:10.481316 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.481296    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-mmpgv\""
Jul 23 23:14:10.481416 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.481342    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 23 23:14:10.487789 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.487764    2578 generic.go:356] "Generic (PLEG): container finished" podID="18ba94f3-1bd7-41f4-a603-0a2654475a66" containerID="e786ad6b7390e8894b3291564b0f27d5fee971599a14fce46fc1dbecc7c1a3d9" exitCode=0
Jul 23 23:14:10.487904 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.487805    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rw58q" event={"ID":"18ba94f3-1bd7-41f4-a603-0a2654475a66","Type":"ContainerDied","Data":"e786ad6b7390e8894b3291564b0f27d5fee971599a14fce46fc1dbecc7c1a3d9"}
Jul 23 23:14:10.493204 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.493180    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hsq95"]
Jul 23 23:14:10.555699 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.555647    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b7125927-8aa3-4326-b1f2-8950ca674b39-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6pgdz\" (UID: \"b7125927-8aa3-4326-b1f2-8950ca674b39\") " pod="openshift-insights/insights-runtime-extractor-6pgdz"
Jul 23 23:14:10.555864 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.555706    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b7125927-8aa3-4326-b1f2-8950ca674b39-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6pgdz\" (UID: \"b7125927-8aa3-4326-b1f2-8950ca674b39\") " pod="openshift-insights/insights-runtime-extractor-6pgdz"
Jul 23 23:14:10.555864 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.555729    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b7125927-8aa3-4326-b1f2-8950ca674b39-data-volume\") pod \"insights-runtime-extractor-6pgdz\" (UID: \"b7125927-8aa3-4326-b1f2-8950ca674b39\") " pod="openshift-insights/insights-runtime-extractor-6pgdz"
Jul 23 23:14:10.555864 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.555745    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0171d6e4-8fda-499c-83a3-b0ec8ec597d1-config-volume\") pod \"dns-default-9bpqj\" (UID: \"0171d6e4-8fda-499c-83a3-b0ec8ec597d1\") " pod="openshift-dns/dns-default-9bpqj"
Jul 23 23:14:10.555864 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.555777    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0171d6e4-8fda-499c-83a3-b0ec8ec597d1-tmp-dir\") pod \"dns-default-9bpqj\" (UID: \"0171d6e4-8fda-499c-83a3-b0ec8ec597d1\") " pod="openshift-dns/dns-default-9bpqj"
Jul 23 23:14:10.555864 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.555799    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b7125927-8aa3-4326-b1f2-8950ca674b39-crio-socket\") pod \"insights-runtime-extractor-6pgdz\" (UID: \"b7125927-8aa3-4326-b1f2-8950ca674b39\") " pod="openshift-insights/insights-runtime-extractor-6pgdz"
Jul 23 23:14:10.555864 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.555821    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45zrr\" (UniqueName: \"kubernetes.io/projected/0171d6e4-8fda-499c-83a3-b0ec8ec597d1-kube-api-access-45zrr\") pod \"dns-default-9bpqj\" (UID: \"0171d6e4-8fda-499c-83a3-b0ec8ec597d1\") " pod="openshift-dns/dns-default-9bpqj"
Jul 23 23:14:10.556055 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.555871    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfln8\" (UniqueName: \"kubernetes.io/projected/b7125927-8aa3-4326-b1f2-8950ca674b39-kube-api-access-cfln8\") pod \"insights-runtime-extractor-6pgdz\" (UID: \"b7125927-8aa3-4326-b1f2-8950ca674b39\") " pod="openshift-insights/insights-runtime-extractor-6pgdz"
Jul 23 23:14:10.556055 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.555892    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0171d6e4-8fda-499c-83a3-b0ec8ec597d1-metrics-tls\") pod \"dns-default-9bpqj\" (UID: \"0171d6e4-8fda-499c-83a3-b0ec8ec597d1\") " pod="openshift-dns/dns-default-9bpqj"
Jul 23 23:14:10.656916 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.656812    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfln8\" (UniqueName: \"kubernetes.io/projected/b7125927-8aa3-4326-b1f2-8950ca674b39-kube-api-access-cfln8\") pod \"insights-runtime-extractor-6pgdz\" (UID: \"b7125927-8aa3-4326-b1f2-8950ca674b39\") " pod="openshift-insights/insights-runtime-extractor-6pgdz"
Jul 23 23:14:10.656916 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.656854    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2d8p9\" (UniqueName: \"kubernetes.io/projected/db753c50-9066-482f-88d2-94dc1b55de52-kube-api-access-2d8p9\") pod \"ingress-canary-hsq95\" (UID: \"db753c50-9066-482f-88d2-94dc1b55de52\") " pod="openshift-ingress-canary/ingress-canary-hsq95"
Jul 23 23:14:10.656916 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.656894    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0171d6e4-8fda-499c-83a3-b0ec8ec597d1-metrics-tls\") pod \"dns-default-9bpqj\" (UID: \"0171d6e4-8fda-499c-83a3-b0ec8ec597d1\") " pod="openshift-dns/dns-default-9bpqj"
Jul 23 23:14:10.657202 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.656974    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b7125927-8aa3-4326-b1f2-8950ca674b39-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6pgdz\" (UID: \"b7125927-8aa3-4326-b1f2-8950ca674b39\") " pod="openshift-insights/insights-runtime-extractor-6pgdz"
Jul 23 23:14:10.657202 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.657006    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b7125927-8aa3-4326-b1f2-8950ca674b39-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6pgdz\" (UID: \"b7125927-8aa3-4326-b1f2-8950ca674b39\") " pod="openshift-insights/insights-runtime-extractor-6pgdz"
Jul 23 23:14:10.657202 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.657034    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b7125927-8aa3-4326-b1f2-8950ca674b39-data-volume\") pod \"insights-runtime-extractor-6pgdz\" (UID: \"b7125927-8aa3-4326-b1f2-8950ca674b39\") " pod="openshift-insights/insights-runtime-extractor-6pgdz"
Jul 23 23:14:10.657202 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.657054    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0171d6e4-8fda-499c-83a3-b0ec8ec597d1-config-volume\") pod \"dns-default-9bpqj\" (UID: \"0171d6e4-8fda-499c-83a3-b0ec8ec597d1\") " pod="openshift-dns/dns-default-9bpqj"
Jul 23 23:14:10.657202 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.657094    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0171d6e4-8fda-499c-83a3-b0ec8ec597d1-tmp-dir\") pod \"dns-default-9bpqj\" (UID: \"0171d6e4-8fda-499c-83a3-b0ec8ec597d1\") " pod="openshift-dns/dns-default-9bpqj"
Jul 23 23:14:10.657202 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.657123    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b7125927-8aa3-4326-b1f2-8950ca674b39-crio-socket\") pod \"insights-runtime-extractor-6pgdz\" (UID: \"b7125927-8aa3-4326-b1f2-8950ca674b39\") " pod="openshift-insights/insights-runtime-extractor-6pgdz"
Jul 23 23:14:10.657202 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.657146    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45zrr\" (UniqueName: \"kubernetes.io/projected/0171d6e4-8fda-499c-83a3-b0ec8ec597d1-kube-api-access-45zrr\") pod \"dns-default-9bpqj\" (UID: \"0171d6e4-8fda-499c-83a3-b0ec8ec597d1\") " pod="openshift-dns/dns-default-9bpqj"
Jul 23 23:14:10.657202 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.657178    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/db753c50-9066-482f-88d2-94dc1b55de52-cert\") pod \"ingress-canary-hsq95\" (UID: \"db753c50-9066-482f-88d2-94dc1b55de52\") " pod="openshift-ingress-canary/ingress-canary-hsq95"
Jul 23 23:14:10.657528 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.657445    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b7125927-8aa3-4326-b1f2-8950ca674b39-crio-socket\") pod \"insights-runtime-extractor-6pgdz\" (UID: \"b7125927-8aa3-4326-b1f2-8950ca674b39\") " pod="openshift-insights/insights-runtime-extractor-6pgdz"
Jul 23 23:14:10.657528 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.657448    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b7125927-8aa3-4326-b1f2-8950ca674b39-data-volume\") pod \"insights-runtime-extractor-6pgdz\" (UID: \"b7125927-8aa3-4326-b1f2-8950ca674b39\") " pod="openshift-insights/insights-runtime-extractor-6pgdz"
Jul 23 23:14:10.657689 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.657636    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b7125927-8aa3-4326-b1f2-8950ca674b39-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6pgdz\" (UID: \"b7125927-8aa3-4326-b1f2-8950ca674b39\") " pod="openshift-insights/insights-runtime-extractor-6pgdz"
Jul 23 23:14:10.657689 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.657685    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0171d6e4-8fda-499c-83a3-b0ec8ec597d1-config-volume\") pod \"dns-default-9bpqj\" (UID: \"0171d6e4-8fda-499c-83a3-b0ec8ec597d1\") " pod="openshift-dns/dns-default-9bpqj"
Jul 23 23:14:10.657816 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.657713    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0171d6e4-8fda-499c-83a3-b0ec8ec597d1-tmp-dir\") pod \"dns-default-9bpqj\" (UID: \"0171d6e4-8fda-499c-83a3-b0ec8ec597d1\") " pod="openshift-dns/dns-default-9bpqj"
Jul 23 23:14:10.661220 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.661197    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b7125927-8aa3-4326-b1f2-8950ca674b39-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6pgdz\" (UID: \"b7125927-8aa3-4326-b1f2-8950ca674b39\") " pod="openshift-insights/insights-runtime-extractor-6pgdz"
Jul 23 23:14:10.661338 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.661204    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0171d6e4-8fda-499c-83a3-b0ec8ec597d1-metrics-tls\") pod \"dns-default-9bpqj\" (UID: \"0171d6e4-8fda-499c-83a3-b0ec8ec597d1\") " pod="openshift-dns/dns-default-9bpqj"
Jul 23 23:14:10.666823 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.666799    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfln8\" (UniqueName: \"kubernetes.io/projected/b7125927-8aa3-4326-b1f2-8950ca674b39-kube-api-access-cfln8\") pod \"insights-runtime-extractor-6pgdz\" (UID: \"b7125927-8aa3-4326-b1f2-8950ca674b39\") " pod="openshift-insights/insights-runtime-extractor-6pgdz"
Jul 23 23:14:10.666941 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.666871    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45zrr\" (UniqueName: \"kubernetes.io/projected/0171d6e4-8fda-499c-83a3-b0ec8ec597d1-kube-api-access-45zrr\") pod \"dns-default-9bpqj\" (UID: \"0171d6e4-8fda-499c-83a3-b0ec8ec597d1\") " pod="openshift-dns/dns-default-9bpqj"
Jul 23 23:14:10.758079 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.758041    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/db753c50-9066-482f-88d2-94dc1b55de52-cert\") pod \"ingress-canary-hsq95\" (UID: \"db753c50-9066-482f-88d2-94dc1b55de52\") " pod="openshift-ingress-canary/ingress-canary-hsq95"
Jul 23 23:14:10.758219 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.758093    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2d8p9\" (UniqueName: \"kubernetes.io/projected/db753c50-9066-482f-88d2-94dc1b55de52-kube-api-access-2d8p9\") pod \"ingress-canary-hsq95\" (UID: \"db753c50-9066-482f-88d2-94dc1b55de52\") " pod="openshift-ingress-canary/ingress-canary-hsq95"
Jul 23 23:14:10.760815 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.760789    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/db753c50-9066-482f-88d2-94dc1b55de52-cert\") pod \"ingress-canary-hsq95\" (UID: \"db753c50-9066-482f-88d2-94dc1b55de52\") " pod="openshift-ingress-canary/ingress-canary-hsq95"
Jul 23 23:14:10.766483 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.766454    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2d8p9\" (UniqueName: \"kubernetes.io/projected/db753c50-9066-482f-88d2-94dc1b55de52-kube-api-access-2d8p9\") pod \"ingress-canary-hsq95\" (UID: \"db753c50-9066-482f-88d2-94dc1b55de52\") " pod="openshift-ingress-canary/ingress-canary-hsq95"
Jul 23 23:14:10.766579 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.766572    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9bpqj"
Jul 23 23:14:10.773101 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.773078    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6pgdz"
Jul 23 23:14:10.786883 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.786852    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hsq95"
Jul 23 23:14:10.941188 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.941154    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9bpqj"]
Jul 23 23:14:10.944236 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:14:10.944208    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0171d6e4_8fda_499c_83a3_b0ec8ec597d1.slice/crio-9ea2c8aec4fca56692730460c491019ccfbc65f464a4fdf8c39171e0926ded89 WatchSource:0}: Error finding container 9ea2c8aec4fca56692730460c491019ccfbc65f464a4fdf8c39171e0926ded89: Status 404 returned error can't find the container with id 9ea2c8aec4fca56692730460c491019ccfbc65f464a4fdf8c39171e0926ded89
Jul 23 23:14:10.955106 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.955050    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hsq95"]
Jul 23 23:14:10.955919 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:10.955893    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6pgdz"]
Jul 23 23:14:10.959511 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:14:10.959485    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7125927_8aa3_4326_b1f2_8950ca674b39.slice/crio-be285bcb9afb3b7a54b4a6b9e56e67174aea02fbb7709dcb6c3ee44a49c00523 WatchSource:0}: Error finding container be285bcb9afb3b7a54b4a6b9e56e67174aea02fbb7709dcb6c3ee44a49c00523: Status 404 returned error can't find the container with id be285bcb9afb3b7a54b4a6b9e56e67174aea02fbb7709dcb6c3ee44a49c00523
Jul 23 23:14:10.960046 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:14:10.959909    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb753c50_9066_482f_88d2_94dc1b55de52.slice/crio-35ff41372ecea87631e889415765743281a6930dd4164ab95297726ecc9fb53f WatchSource:0}: Error finding container 35ff41372ecea87631e889415765743281a6930dd4164ab95297726ecc9fb53f: Status 404 returned error can't find the container with id 35ff41372ecea87631e889415765743281a6930dd4164ab95297726ecc9fb53f
Jul 23 23:14:11.232845 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:11.232755    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5d9sj"
Jul 23 23:14:11.232845 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:11.232794    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bbv68"
Jul 23 23:14:11.233069 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:11.232868    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z8d42"
Jul 23 23:14:11.235818 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:11.235792    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 23 23:14:11.235935 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:11.235884    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 23 23:14:11.237087 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:11.237049    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-86k6d\""
Jul 23 23:14:11.237087 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:11.237066    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 23 23:14:11.237087 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:11.237059    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 23 23:14:11.237271 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:11.237110    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rl54t\""
Jul 23 23:14:11.491912 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:11.491814    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hsq95" event={"ID":"db753c50-9066-482f-88d2-94dc1b55de52","Type":"ContainerStarted","Data":"35ff41372ecea87631e889415765743281a6930dd4164ab95297726ecc9fb53f"}
Jul 23 23:14:11.493174 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:11.493141    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6pgdz" event={"ID":"b7125927-8aa3-4326-b1f2-8950ca674b39","Type":"ContainerStarted","Data":"be285bcb9afb3b7a54b4a6b9e56e67174aea02fbb7709dcb6c3ee44a49c00523"}
Jul 23 23:14:11.497304 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:11.497267    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rw58q" event={"ID":"18ba94f3-1bd7-41f4-a603-0a2654475a66","Type":"ContainerStarted","Data":"ca2c812a1e6e5029a34672b70930424e8ea84afaac5bdbf91fd793f3b17d4926"}
Jul 23 23:14:11.498644 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:11.498601    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9bpqj" event={"ID":"0171d6e4-8fda-499c-83a3-b0ec8ec597d1","Type":"ContainerStarted","Data":"9ea2c8aec4fca56692730460c491019ccfbc65f464a4fdf8c39171e0926ded89"}
Jul 23 23:14:11.519097 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:11.519043    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-rw58q" podStartSLOduration=5.981316982 podStartE2EDuration="39.519022479s" totalImagesPullingTime="33.537705497s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 23:13:32 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:14:11.517274225 +0000 UTC m=+40.886799888" watchObservedRunningTime="2026-07-23 23:14:11.519022479 +0000 UTC m=+40.888548137"
Jul 23 23:14:14.126547 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:14.126504    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-84cf6d97f9-wnz99"]
Jul 23 23:14:14.148631 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:14.148597    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84cf6d97f9-wnz99"]
Jul 23 23:14:14.148809 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:14.148761    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84cf6d97f9-wnz99"
Jul 23 23:14:14.151483 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:14.151450    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 23 23:14:14.152361 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:14.151975    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 23 23:14:14.152361 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:14.152197    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 23 23:14:14.153236 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:14.152843    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-njw8z\""
Jul 23 23:14:14.153841 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:14.153822    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 23 23:14:14.154483 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:14.154470    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 23 23:14:14.154944 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:14.154913    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 23 23:14:14.156677 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:14.155129    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 23 23:14:14.170236 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:14.170208    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 23 23:14:14.288804 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:14.288768    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c6506011-aa7d-49d6-8086-a3d097207fc0-service-ca\") pod \"console-84cf6d97f9-wnz99\" (UID: \"c6506011-aa7d-49d6-8086-a3d097207fc0\") " pod="openshift-console/console-84cf6d97f9-wnz99"
Jul 23 23:14:14.288978 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:14.288819    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c6506011-aa7d-49d6-8086-a3d097207fc0-console-serving-cert\") pod \"console-84cf6d97f9-wnz99\" (UID: \"c6506011-aa7d-49d6-8086-a3d097207fc0\") " pod="openshift-console/console-84cf6d97f9-wnz99"
Jul 23 23:14:14.288978 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:14.288912    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6506011-aa7d-49d6-8086-a3d097207fc0-trusted-ca-bundle\") pod \"console-84cf6d97f9-wnz99\" (UID: \"c6506011-aa7d-49d6-8086-a3d097207fc0\") " pod="openshift-console/console-84cf6d97f9-wnz99"
Jul 23 23:14:14.288978 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:14.288955    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c6506011-aa7d-49d6-8086-a3d097207fc0-console-config\") pod \"console-84cf6d97f9-wnz99\" (UID: \"c6506011-aa7d-49d6-8086-a3d097207fc0\") " pod="openshift-console/console-84cf6d97f9-wnz99"
Jul 23 23:14:14.289080 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:14.289002    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kcpj\" (UniqueName: \"kubernetes.io/projected/c6506011-aa7d-49d6-8086-a3d097207fc0-kube-api-access-8kcpj\") pod \"console-84cf6d97f9-wnz99\" (UID: \"c6506011-aa7d-49d6-8086-a3d097207fc0\") " pod="openshift-console/console-84cf6d97f9-wnz99"
Jul 23 23:14:14.289080 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:14.289026    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c6506011-aa7d-49d6-8086-a3d097207fc0-console-oauth-config\") pod \"console-84cf6d97f9-wnz99\" (UID: \"c6506011-aa7d-49d6-8086-a3d097207fc0\") " pod="openshift-console/console-84cf6d97f9-wnz99"
Jul 23 23:14:14.289080 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:14.289041    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c6506011-aa7d-49d6-8086-a3d097207fc0-oauth-serving-cert\") pod \"console-84cf6d97f9-wnz99\" (UID: \"c6506011-aa7d-49d6-8086-a3d097207fc0\") " pod="openshift-console/console-84cf6d97f9-wnz99"
Jul 23 23:14:14.390331 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:14.390235    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6506011-aa7d-49d6-8086-a3d097207fc0-trusted-ca-bundle\") pod \"console-84cf6d97f9-wnz99\" (UID: \"c6506011-aa7d-49d6-8086-a3d097207fc0\") " pod="openshift-console/console-84cf6d97f9-wnz99"
Jul 23 23:14:14.390331 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:14.390283    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c6506011-aa7d-49d6-8086-a3d097207fc0-console-config\") pod \"console-84cf6d97f9-wnz99\" (UID: \"c6506011-aa7d-49d6-8086-a3d097207fc0\") " pod="openshift-console/console-84cf6d97f9-wnz99"
Jul 23 23:14:14.390534 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:14.390380    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kcpj\" (UniqueName: \"kubernetes.io/projected/c6506011-aa7d-49d6-8086-a3d097207fc0-kube-api-access-8kcpj\") pod \"console-84cf6d97f9-wnz99\" (UID: \"c6506011-aa7d-49d6-8086-a3d097207fc0\") " pod="openshift-console/console-84cf6d97f9-wnz99"
Jul 23 23:14:14.390534 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:14.390428    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c6506011-aa7d-49d6-8086-a3d097207fc0-console-oauth-config\") pod \"console-84cf6d97f9-wnz99\" (UID: \"c6506011-aa7d-49d6-8086-a3d097207fc0\") " pod="openshift-console/console-84cf6d97f9-wnz99"
Jul 23 23:14:14.390534 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:14.390461    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c6506011-aa7d-49d6-8086-a3d097207fc0-oauth-serving-cert\") pod \"console-84cf6d97f9-wnz99\" (UID: \"c6506011-aa7d-49d6-8086-a3d097207fc0\") " pod="openshift-console/console-84cf6d97f9-wnz99"
Jul 23 23:14:14.390730 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:14.390691    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c6506011-aa7d-49d6-8086-a3d097207fc0-service-ca\") pod \"console-84cf6d97f9-wnz99\" (UID: \"c6506011-aa7d-49d6-8086-a3d097207fc0\") " pod="openshift-console/console-84cf6d97f9-wnz99"
Jul 23 23:14:14.390782 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:14.390764    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c6506011-aa7d-49d6-8086-a3d097207fc0-console-serving-cert\") pod \"console-84cf6d97f9-wnz99\" (UID: \"c6506011-aa7d-49d6-8086-a3d097207fc0\") " pod="openshift-console/console-84cf6d97f9-wnz99"
Jul 23 23:14:14.391321 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:14.391297    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c6506011-aa7d-49d6-8086-a3d097207fc0-console-config\") pod \"console-84cf6d97f9-wnz99\" (UID: \"c6506011-aa7d-49d6-8086-a3d097207fc0\") " pod="openshift-console/console-84cf6d97f9-wnz99"
Jul 23 23:14:14.391321 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:14.391312    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c6506011-aa7d-49d6-8086-a3d097207fc0-oauth-serving-cert\") pod \"console-84cf6d97f9-wnz99\" (UID: \"c6506011-aa7d-49d6-8086-a3d097207fc0\") " pod="openshift-console/console-84cf6d97f9-wnz99"
Jul 23 23:14:14.391480 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:14.391428    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c6506011-aa7d-49d6-8086-a3d097207fc0-service-ca\") pod \"console-84cf6d97f9-wnz99\" (UID: \"c6506011-aa7d-49d6-8086-a3d097207fc0\") " pod="openshift-console/console-84cf6d97f9-wnz99"
Jul 23 23:14:14.392035 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:14.392011    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6506011-aa7d-49d6-8086-a3d097207fc0-trusted-ca-bundle\") pod \"console-84cf6d97f9-wnz99\" (UID: \"c6506011-aa7d-49d6-8086-a3d097207fc0\") " pod="openshift-console/console-84cf6d97f9-wnz99"
Jul 23 23:14:14.393318 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:14.393297    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c6506011-aa7d-49d6-8086-a3d097207fc0-console-serving-cert\") pod \"console-84cf6d97f9-wnz99\" (UID: \"c6506011-aa7d-49d6-8086-a3d097207fc0\") " pod="openshift-console/console-84cf6d97f9-wnz99"
Jul 23 23:14:14.393428 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:14.393405    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c6506011-aa7d-49d6-8086-a3d097207fc0-console-oauth-config\") pod \"console-84cf6d97f9-wnz99\" (UID: \"c6506011-aa7d-49d6-8086-a3d097207fc0\") " pod="openshift-console/console-84cf6d97f9-wnz99"
Jul 23 23:14:14.399315 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:14.399285    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kcpj\" (UniqueName: \"kubernetes.io/projected/c6506011-aa7d-49d6-8086-a3d097207fc0-kube-api-access-8kcpj\") pod \"console-84cf6d97f9-wnz99\" (UID: \"c6506011-aa7d-49d6-8086-a3d097207fc0\") " pod="openshift-console/console-84cf6d97f9-wnz99"
Jul 23 23:14:14.463189 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:14.463148    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84cf6d97f9-wnz99"
Jul 23 23:14:17.821695 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:17.821666    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84cf6d97f9-wnz99"]
Jul 23 23:14:17.825514 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:14:17.825480    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6506011_aa7d_49d6_8086_a3d097207fc0.slice/crio-482d48404b8ae0719126970ab47446a92dc28207fc57c8c8678f9659fca3b980 WatchSource:0}: Error finding container 482d48404b8ae0719126970ab47446a92dc28207fc57c8c8678f9659fca3b980: Status 404 returned error can't find the container with id 482d48404b8ae0719126970ab47446a92dc28207fc57c8c8678f9659fca3b980
Jul 23 23:14:18.519769 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:18.519724    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6pgdz" event={"ID":"b7125927-8aa3-4326-b1f2-8950ca674b39","Type":"ContainerStarted","Data":"e8a5c9cecf4cd8e489d6a17eedb4e34468dbd6aa9caf78280729d925548f5f98"}
Jul 23 23:14:18.523647 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:18.523611    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9bpqj" event={"ID":"0171d6e4-8fda-499c-83a3-b0ec8ec597d1","Type":"ContainerStarted","Data":"a963a213182ddb563ac7796804c877416ccf69298995355141b31fb598d1986b"}
Jul 23 23:14:18.523840 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:18.523741    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9bpqj" event={"ID":"0171d6e4-8fda-499c-83a3-b0ec8ec597d1","Type":"ContainerStarted","Data":"6e03547df9247c81b62ce8d916155f7e70ad71597119010ad3e9a35854ebef38"}
Jul 23 23:14:18.523840 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:18.523791    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9bpqj"
Jul 23 23:14:18.525528 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:18.525500    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hsq95" event={"ID":"db753c50-9066-482f-88d2-94dc1b55de52","Type":"ContainerStarted","Data":"50772421e8e774b5b42ea1799c34076c590c4505da347ec62f1bf6ee75cbc8d5"}
Jul 23 23:14:18.527623 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:18.527592    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84cf6d97f9-wnz99" event={"ID":"c6506011-aa7d-49d6-8086-a3d097207fc0","Type":"ContainerStarted","Data":"482d48404b8ae0719126970ab47446a92dc28207fc57c8c8678f9659fca3b980"}
Jul 23 23:14:18.565390 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:18.565333    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-9bpqj" podStartSLOduration=1.822955063 podStartE2EDuration="8.565311091s" totalImagesPullingTime="6.742356028s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:14:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:14:18.542331661 +0000 UTC m=+47.911857331" watchObservedRunningTime="2026-07-23 23:14:18.565311091 +0000 UTC m=+47.934836727"
Jul 23 23:14:18.565932 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:18.565890    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hsq95" podStartSLOduration=1.836749807 podStartE2EDuration="8.565871743s" totalImagesPullingTime="6.729121936s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:14:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:14:18.565468146 +0000 UTC m=+47.934993785" watchObservedRunningTime="2026-07-23 23:14:18.565871743 +0000 UTC m=+47.935397404"
Jul 23 23:14:18.603324 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:18.603292    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-plx4r"]
Jul 23 23:14:18.616881 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:18.616847    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-plx4r"]
Jul 23 23:14:18.617052 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:18.616954    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-plx4r"
Jul 23 23:14:18.620809 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:18.620781    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-5vn9n\""
Jul 23 23:14:18.621064 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:18.621045    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 23 23:14:18.624263 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:18.624232    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/3d2457d7-2cbe-4d01-ae77-d89c4a280df2-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-plx4r\" (UID: \"3d2457d7-2cbe-4d01-ae77-d89c4a280df2\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-plx4r"
Jul 23 23:14:18.725332 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:18.725284    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/3d2457d7-2cbe-4d01-ae77-d89c4a280df2-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-plx4r\" (UID: \"3d2457d7-2cbe-4d01-ae77-d89c4a280df2\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-plx4r"
Jul 23 23:14:18.725512 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:14:18.725450    2578 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 23 23:14:18.725573 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:14:18.725517    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d2457d7-2cbe-4d01-ae77-d89c4a280df2-tls-certificates podName:3d2457d7-2cbe-4d01-ae77-d89c4a280df2 nodeName:}" failed. No retries permitted until 2026-07-23 23:14:19.225496965 +0000 UTC m=+48.595022625 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/3d2457d7-2cbe-4d01-ae77-d89c4a280df2-tls-certificates") pod "prometheus-operator-admission-webhook-7f8d78f54b-plx4r" (UID: "3d2457d7-2cbe-4d01-ae77-d89c4a280df2") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 23 23:14:19.229268 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:19.229233    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/3d2457d7-2cbe-4d01-ae77-d89c4a280df2-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-plx4r\" (UID: \"3d2457d7-2cbe-4d01-ae77-d89c4a280df2\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-plx4r"
Jul 23 23:14:19.234906 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:19.234881    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/3d2457d7-2cbe-4d01-ae77-d89c4a280df2-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-plx4r\" (UID: \"3d2457d7-2cbe-4d01-ae77-d89c4a280df2\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-plx4r"
Jul 23 23:14:19.235084 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:19.234978    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-plx4r"
Jul 23 23:14:19.422038 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:19.420904    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-plx4r"]
Jul 23 23:14:19.425570 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:14:19.425540    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d2457d7_2cbe_4d01_ae77_d89c4a280df2.slice/crio-84c52c835afba96ad0c12a737f354b28ed015ce1de0f8fc0e260ce82e0a8086f WatchSource:0}: Error finding container 84c52c835afba96ad0c12a737f354b28ed015ce1de0f8fc0e260ce82e0a8086f: Status 404 returned error can't find the container with id 84c52c835afba96ad0c12a737f354b28ed015ce1de0f8fc0e260ce82e0a8086f
Jul 23 23:14:19.533625 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:19.533536    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-plx4r" event={"ID":"3d2457d7-2cbe-4d01-ae77-d89c4a280df2","Type":"ContainerStarted","Data":"84c52c835afba96ad0c12a737f354b28ed015ce1de0f8fc0e260ce82e0a8086f"}
Jul 23 23:14:25.551127 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:25.551059    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84cf6d97f9-wnz99" event={"ID":"c6506011-aa7d-49d6-8086-a3d097207fc0","Type":"ContainerStarted","Data":"9f148f83eb6144140f26ae6a72235c05b71a537d34c19d2067ff206720a339b9"}
Jul 23 23:14:25.553061 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:25.553027    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-plx4r" event={"ID":"3d2457d7-2cbe-4d01-ae77-d89c4a280df2","Type":"ContainerStarted","Data":"b7f2ce1832cb97ba01efb4fda211313ad31e37c163f9e811981d46dea210e580"}
Jul 23 23:14:25.553204 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:25.553187    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-plx4r"
Jul 23 23:14:25.554956 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:25.554920    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6pgdz" event={"ID":"b7125927-8aa3-4326-b1f2-8950ca674b39","Type":"ContainerStarted","Data":"1ba649bcb02cf13d28d32344933156ef84331146954139e34afe1575140c79b0"}
Jul 23 23:14:25.558944 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:25.558919    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-plx4r"
Jul 23 23:14:25.569269 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:25.569214    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-84cf6d97f9-wnz99" podStartSLOduration=4.89029412 podStartE2EDuration="11.569195537s" totalImagesPullingTime="6.678901417s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:14:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:14:25.568206447 +0000 UTC m=+54.937732119" watchObservedRunningTime="2026-07-23 23:14:25.569195537 +0000 UTC m=+54.938721198"
Jul 23 23:14:25.582676 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:25.582594    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-plx4r" podStartSLOduration=2.505012683 podStartE2EDuration="7.582576735s" totalImagesPullingTime="5.077564052s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:14:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:14:25.582468403 +0000 UTC m=+54.951994065" watchObservedRunningTime="2026-07-23 23:14:25.582576735 +0000 UTC m=+54.952102394"
Jul 23 23:14:26.658025 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:26.657989    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-dp2w9"]
Jul 23 23:14:26.673673 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:26.673629    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-dp2w9"]
Jul 23 23:14:26.673845 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:26.673778    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-dp2w9"
Jul 23 23:14:26.676960 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:26.676937    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 23 23:14:26.678297 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:26.678273    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 23 23:14:26.678423 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:26.678299    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-ptt5q\""
Jul 23 23:14:26.678423 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:26.678316    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 23 23:14:26.678423 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:26.678300    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 23 23:14:26.678423 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:26.678379    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 23 23:14:26.796671 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:26.796617    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0fcfa80a-1df2-4525-8b6e-2ca734dcead8-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-dp2w9\" (UID: \"0fcfa80a-1df2-4525-8b6e-2ca734dcead8\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-dp2w9"
Jul 23 23:14:26.796848 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:26.796713    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0fcfa80a-1df2-4525-8b6e-2ca734dcead8-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-dp2w9\" (UID: \"0fcfa80a-1df2-4525-8b6e-2ca734dcead8\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-dp2w9"
Jul 23 23:14:26.796848 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:26.796747    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0fcfa80a-1df2-4525-8b6e-2ca734dcead8-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-dp2w9\" (UID: \"0fcfa80a-1df2-4525-8b6e-2ca734dcead8\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-dp2w9"
Jul 23 23:14:26.796848 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:26.796773    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrrd6\" (UniqueName: \"kubernetes.io/projected/0fcfa80a-1df2-4525-8b6e-2ca734dcead8-kube-api-access-mrrd6\") pod \"prometheus-operator-84d69f794d-dp2w9\" (UID: \"0fcfa80a-1df2-4525-8b6e-2ca734dcead8\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-dp2w9"
Jul 23 23:14:26.897951 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:26.897920    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0fcfa80a-1df2-4525-8b6e-2ca734dcead8-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-dp2w9\" (UID: \"0fcfa80a-1df2-4525-8b6e-2ca734dcead8\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-dp2w9"
Jul 23 23:14:26.898124 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:26.897969    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0fcfa80a-1df2-4525-8b6e-2ca734dcead8-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-dp2w9\" (UID: \"0fcfa80a-1df2-4525-8b6e-2ca734dcead8\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-dp2w9"
Jul 23 23:14:26.898124 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:26.897999    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0fcfa80a-1df2-4525-8b6e-2ca734dcead8-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-dp2w9\" (UID: \"0fcfa80a-1df2-4525-8b6e-2ca734dcead8\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-dp2w9"
Jul 23 23:14:26.898124 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:14:26.898093    2578 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 23 23:14:26.898124 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:26.898115    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrrd6\" (UniqueName: \"kubernetes.io/projected/0fcfa80a-1df2-4525-8b6e-2ca734dcead8-kube-api-access-mrrd6\") pod \"prometheus-operator-84d69f794d-dp2w9\" (UID: \"0fcfa80a-1df2-4525-8b6e-2ca734dcead8\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-dp2w9"
Jul 23 23:14:26.898306 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:14:26.898170    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fcfa80a-1df2-4525-8b6e-2ca734dcead8-prometheus-operator-tls podName:0fcfa80a-1df2-4525-8b6e-2ca734dcead8 nodeName:}" failed. No retries permitted until 2026-07-23 23:14:27.398146205 +0000 UTC m=+56.767671859 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/0fcfa80a-1df2-4525-8b6e-2ca734dcead8-prometheus-operator-tls") pod "prometheus-operator-84d69f794d-dp2w9" (UID: "0fcfa80a-1df2-4525-8b6e-2ca734dcead8") : secret "prometheus-operator-tls" not found
Jul 23 23:14:26.898800 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:26.898779    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0fcfa80a-1df2-4525-8b6e-2ca734dcead8-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-dp2w9\" (UID: \"0fcfa80a-1df2-4525-8b6e-2ca734dcead8\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-dp2w9"
Jul 23 23:14:26.900798 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:26.900774    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0fcfa80a-1df2-4525-8b6e-2ca734dcead8-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-dp2w9\" (UID: \"0fcfa80a-1df2-4525-8b6e-2ca734dcead8\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-dp2w9"
Jul 23 23:14:26.907306 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:26.907285    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrrd6\" (UniqueName: \"kubernetes.io/projected/0fcfa80a-1df2-4525-8b6e-2ca734dcead8-kube-api-access-mrrd6\") pod \"prometheus-operator-84d69f794d-dp2w9\" (UID: \"0fcfa80a-1df2-4525-8b6e-2ca734dcead8\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-dp2w9"
Jul 23 23:14:27.402549 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:27.402512    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0fcfa80a-1df2-4525-8b6e-2ca734dcead8-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-dp2w9\" (UID: \"0fcfa80a-1df2-4525-8b6e-2ca734dcead8\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-dp2w9"
Jul 23 23:14:27.402762 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:14:27.402715    2578 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 23 23:14:27.402833 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:14:27.402798    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fcfa80a-1df2-4525-8b6e-2ca734dcead8-prometheus-operator-tls podName:0fcfa80a-1df2-4525-8b6e-2ca734dcead8 nodeName:}" failed. No retries permitted until 2026-07-23 23:14:28.402777631 +0000 UTC m=+57.772303270 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/0fcfa80a-1df2-4525-8b6e-2ca734dcead8-prometheus-operator-tls") pod "prometheus-operator-84d69f794d-dp2w9" (UID: "0fcfa80a-1df2-4525-8b6e-2ca734dcead8") : secret "prometheus-operator-tls" not found
Jul 23 23:14:28.410475 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:28.410432    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0fcfa80a-1df2-4525-8b6e-2ca734dcead8-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-dp2w9\" (UID: \"0fcfa80a-1df2-4525-8b6e-2ca734dcead8\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-dp2w9"
Jul 23 23:14:28.412916 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:28.412887    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0fcfa80a-1df2-4525-8b6e-2ca734dcead8-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-dp2w9\" (UID: \"0fcfa80a-1df2-4525-8b6e-2ca734dcead8\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-dp2w9"
Jul 23 23:14:28.481959 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:28.481936    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-dp2w9"
Jul 23 23:14:28.536132 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:28.536038    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9bpqj"
Jul 23 23:14:28.568678 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:28.567995    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6pgdz" event={"ID":"b7125927-8aa3-4326-b1f2-8950ca674b39","Type":"ContainerStarted","Data":"4000883ddbefa2a72c75c4fb8d8430ac80a278e14246f53fc81689550e64088d"}
Jul 23 23:14:28.593863 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:28.590608    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-6pgdz" podStartSLOduration=1.357867075 podStartE2EDuration="18.590585852s" totalImagesPullingTime="17.232718777s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 23:14:10 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:14:28.587782522 +0000 UTC m=+57.957308192" watchObservedRunningTime="2026-07-23 23:14:28.590585852 +0000 UTC m=+57.960111511"
Jul 23 23:14:28.612102 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:28.612072    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-dp2w9"]
Jul 23 23:14:28.614492 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:14:28.614469    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fcfa80a_1df2_4525_8b6e_2ca734dcead8.slice/crio-4a52147a13c186fd0d07ee497578e9cbf11609922c913857c97e5ce46e4b5318 WatchSource:0}: Error finding container 4a52147a13c186fd0d07ee497578e9cbf11609922c913857c97e5ce46e4b5318: Status 404 returned error can't find the container with id 4a52147a13c186fd0d07ee497578e9cbf11609922c913857c97e5ce46e4b5318
Jul 23 23:14:29.572822 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:29.572777    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-dp2w9" event={"ID":"0fcfa80a-1df2-4525-8b6e-2ca734dcead8","Type":"ContainerStarted","Data":"4a52147a13c186fd0d07ee497578e9cbf11609922c913857c97e5ce46e4b5318"}
Jul 23 23:14:30.578313 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:30.578277    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-dp2w9" event={"ID":"0fcfa80a-1df2-4525-8b6e-2ca734dcead8","Type":"ContainerStarted","Data":"1d3cf93585c825a661b208bbfb5f719e05643a6cbecaf4180e994067d31bc86b"}
Jul 23 23:14:30.578709 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:30.578330    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-dp2w9" event={"ID":"0fcfa80a-1df2-4525-8b6e-2ca734dcead8","Type":"ContainerStarted","Data":"6bb6765b2012b59870e680f34a01be65f613c3b2e66913a1b8a443de4a860b92"}
Jul 23 23:14:30.594993 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:30.594943    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-dp2w9" podStartSLOduration=3.494246788 podStartE2EDuration="4.594926807s" totalImagesPullingTime="1.100680019s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:14:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:14:30.594744866 +0000 UTC m=+59.964270528" watchObservedRunningTime="2026-07-23 23:14:30.594926807 +0000 UTC m=+59.964452466"
Jul 23 23:14:32.015175 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.015143    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-6c8qv"]
Jul 23 23:14:32.017733 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.017710    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6c8qv"
Jul 23 23:14:32.020311 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.020290    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 23 23:14:32.020416 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.020335    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 23 23:14:32.020462 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.020423    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-5p5mq\""
Jul 23 23:14:32.025348 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.025329    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-v29tt"]
Jul 23 23:14:32.027392 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.027376    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v29tt"
Jul 23 23:14:32.029099 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.029076    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-6c8qv"]
Jul 23 23:14:32.029904 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.029886    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 23 23:14:32.030006 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.029909    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-cp5vq\""
Jul 23 23:14:32.030006 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.029892    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 23 23:14:32.030359 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.030339    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 23 23:14:32.041322 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.041301    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-wcbw2"]
Jul 23 23:14:32.043443 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.043427    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wcbw2"
Jul 23 23:14:32.045113 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.045094    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-v29tt"]
Jul 23 23:14:32.045870 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.045854    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 23 23:14:32.045971 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.045887    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 23 23:14:32.046030 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.046007    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 23 23:14:32.046225 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.046209    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-52bjn\""
Jul 23 23:14:32.138153 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.138119    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/292b6375-d147-4097-8eef-61692f076e3a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-6c8qv\" (UID: \"292b6375-d147-4097-8eef-61692f076e3a\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6c8qv"
Jul 23 23:14:32.138153 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.138153    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8vgw\" (UniqueName: \"kubernetes.io/projected/426f9a19-5c8c-4c1c-aaba-fc30325db699-kube-api-access-s8vgw\") pod \"node-exporter-wcbw2\" (UID: \"426f9a19-5c8c-4c1c-aaba-fc30325db699\") " pod="openshift-monitoring/node-exporter-wcbw2"
Jul 23 23:14:32.138402 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.138176    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/426f9a19-5c8c-4c1c-aaba-fc30325db699-node-exporter-accelerators-collector-config\") pod \"node-exporter-wcbw2\" (UID: \"426f9a19-5c8c-4c1c-aaba-fc30325db699\") " pod="openshift-monitoring/node-exporter-wcbw2"
Jul 23 23:14:32.138402 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.138201    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/9ee7a305-ec45-4c90-9033-05158464f844-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-v29tt\" (UID: \"9ee7a305-ec45-4c90-9033-05158464f844\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v29tt"
Jul 23 23:14:32.138402 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.138258    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/426f9a19-5c8c-4c1c-aaba-fc30325db699-node-exporter-tls\") pod \"node-exporter-wcbw2\" (UID: \"426f9a19-5c8c-4c1c-aaba-fc30325db699\") " pod="openshift-monitoring/node-exporter-wcbw2"
Jul 23 23:14:32.138402 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.138347    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/292b6375-d147-4097-8eef-61692f076e3a-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-6c8qv\" (UID: \"292b6375-d147-4097-8eef-61692f076e3a\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6c8qv"
Jul 23 23:14:32.138402 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.138367    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ee7a305-ec45-4c90-9033-05158464f844-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-v29tt\" (UID: \"9ee7a305-ec45-4c90-9033-05158464f844\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v29tt"
Jul 23 23:14:32.138402 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.138392    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hv8q8\" (UniqueName: \"kubernetes.io/projected/292b6375-d147-4097-8eef-61692f076e3a-kube-api-access-hv8q8\") pod \"openshift-state-metrics-b8f49f5c6-6c8qv\" (UID: \"292b6375-d147-4097-8eef-61692f076e3a\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6c8qv"
Jul 23 23:14:32.138724 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.138422    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/292b6375-d147-4097-8eef-61692f076e3a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-6c8qv\" (UID: \"292b6375-d147-4097-8eef-61692f076e3a\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6c8qv"
Jul 23 23:14:32.138724 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.138438    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/426f9a19-5c8c-4c1c-aaba-fc30325db699-root\") pod \"node-exporter-wcbw2\" (UID: \"426f9a19-5c8c-4c1c-aaba-fc30325db699\") " pod="openshift-monitoring/node-exporter-wcbw2"
Jul 23 23:14:32.138724 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.138468    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sdc7\" (UniqueName: \"kubernetes.io/projected/9ee7a305-ec45-4c90-9033-05158464f844-kube-api-access-4sdc7\") pod \"kube-state-metrics-56d9f944b4-v29tt\" (UID: \"9ee7a305-ec45-4c90-9033-05158464f844\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v29tt"
Jul 23 23:14:32.138724 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.138502    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/426f9a19-5c8c-4c1c-aaba-fc30325db699-node-exporter-textfile\") pod \"node-exporter-wcbw2\" (UID: \"426f9a19-5c8c-4c1c-aaba-fc30325db699\") " pod="openshift-monitoring/node-exporter-wcbw2"
Jul 23 23:14:32.138724 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.138545    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/9ee7a305-ec45-4c90-9033-05158464f844-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-v29tt\" (UID: \"9ee7a305-ec45-4c90-9033-05158464f844\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v29tt"
Jul 23 23:14:32.138724 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.138586    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9ee7a305-ec45-4c90-9033-05158464f844-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-v29tt\" (UID: \"9ee7a305-ec45-4c90-9033-05158464f844\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v29tt"
Jul 23 23:14:32.138724 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.138632    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/426f9a19-5c8c-4c1c-aaba-fc30325db699-node-exporter-wtmp\") pod \"node-exporter-wcbw2\" (UID: \"426f9a19-5c8c-4c1c-aaba-fc30325db699\") " pod="openshift-monitoring/node-exporter-wcbw2"
Jul 23 23:14:32.138724 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.138692    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9ee7a305-ec45-4c90-9033-05158464f844-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-v29tt\" (UID: \"9ee7a305-ec45-4c90-9033-05158464f844\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v29tt"
Jul 23 23:14:32.139036 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.138734    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/426f9a19-5c8c-4c1c-aaba-fc30325db699-metrics-client-ca\") pod \"node-exporter-wcbw2\" (UID: \"426f9a19-5c8c-4c1c-aaba-fc30325db699\") " pod="openshift-monitoring/node-exporter-wcbw2"
Jul 23 23:14:32.139036 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.138803    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/426f9a19-5c8c-4c1c-aaba-fc30325db699-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wcbw2\" (UID: \"426f9a19-5c8c-4c1c-aaba-fc30325db699\") " pod="openshift-monitoring/node-exporter-wcbw2"
Jul 23 23:14:32.139036 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.138828    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/426f9a19-5c8c-4c1c-aaba-fc30325db699-sys\") pod \"node-exporter-wcbw2\" (UID: \"426f9a19-5c8c-4c1c-aaba-fc30325db699\") " pod="openshift-monitoring/node-exporter-wcbw2"
Jul 23 23:14:32.240125 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.240088    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/426f9a19-5c8c-4c1c-aaba-fc30325db699-metrics-client-ca\") pod \"node-exporter-wcbw2\" (UID: \"426f9a19-5c8c-4c1c-aaba-fc30325db699\") " pod="openshift-monitoring/node-exporter-wcbw2"
Jul 23 23:14:32.240304 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.240137    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/426f9a19-5c8c-4c1c-aaba-fc30325db699-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wcbw2\" (UID: \"426f9a19-5c8c-4c1c-aaba-fc30325db699\") " pod="openshift-monitoring/node-exporter-wcbw2"
Jul 23 23:14:32.240304 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.240155    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/426f9a19-5c8c-4c1c-aaba-fc30325db699-sys\") pod \"node-exporter-wcbw2\" (UID: \"426f9a19-5c8c-4c1c-aaba-fc30325db699\") " pod="openshift-monitoring/node-exporter-wcbw2"
Jul 23 23:14:32.240304 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.240185    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/292b6375-d147-4097-8eef-61692f076e3a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-6c8qv\" (UID: \"292b6375-d147-4097-8eef-61692f076e3a\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6c8qv"
Jul 23 23:14:32.240304 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.240211    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8vgw\" (UniqueName: \"kubernetes.io/projected/426f9a19-5c8c-4c1c-aaba-fc30325db699-kube-api-access-s8vgw\") pod \"node-exporter-wcbw2\" (UID: \"426f9a19-5c8c-4c1c-aaba-fc30325db699\") " pod="openshift-monitoring/node-exporter-wcbw2"
Jul 23 23:14:32.240304 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.240234    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/426f9a19-5c8c-4c1c-aaba-fc30325db699-sys\") pod \"node-exporter-wcbw2\" (UID: \"426f9a19-5c8c-4c1c-aaba-fc30325db699\") " pod="openshift-monitoring/node-exporter-wcbw2"
Jul 23 23:14:32.240304 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.240243    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/426f9a19-5c8c-4c1c-aaba-fc30325db699-node-exporter-accelerators-collector-config\") pod \"node-exporter-wcbw2\" (UID: \"426f9a19-5c8c-4c1c-aaba-fc30325db699\") " pod="openshift-monitoring/node-exporter-wcbw2"
Jul 23 23:14:32.240304 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.240281    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/9ee7a305-ec45-4c90-9033-05158464f844-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-v29tt\" (UID: \"9ee7a305-ec45-4c90-9033-05158464f844\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v29tt"
Jul 23 23:14:32.240638 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.240479    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/426f9a19-5c8c-4c1c-aaba-fc30325db699-node-exporter-tls\") pod \"node-exporter-wcbw2\" (UID: \"426f9a19-5c8c-4c1c-aaba-fc30325db699\") " pod="openshift-monitoring/node-exporter-wcbw2"
Jul 23 23:14:32.240638 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.240550    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/292b6375-d147-4097-8eef-61692f076e3a-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-6c8qv\" (UID: \"292b6375-d147-4097-8eef-61692f076e3a\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6c8qv"
Jul 23 23:14:32.240638 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.240579    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ee7a305-ec45-4c90-9033-05158464f844-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-v29tt\" (UID: \"9ee7a305-ec45-4c90-9033-05158464f844\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v29tt"
Jul 23 23:14:32.240638 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:14:32.240597    2578 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 23 23:14:32.240638 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.240625    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hv8q8\" (UniqueName: \"kubernetes.io/projected/292b6375-d147-4097-8eef-61692f076e3a-kube-api-access-hv8q8\") pod \"openshift-state-metrics-b8f49f5c6-6c8qv\" (UID: \"292b6375-d147-4097-8eef-61692f076e3a\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6c8qv"
Jul 23 23:14:32.240919 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:14:32.240704    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/426f9a19-5c8c-4c1c-aaba-fc30325db699-node-exporter-tls podName:426f9a19-5c8c-4c1c-aaba-fc30325db699 nodeName:}" failed. No retries permitted until 2026-07-23 23:14:32.740640096 +0000 UTC m=+62.110165749 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/426f9a19-5c8c-4c1c-aaba-fc30325db699-node-exporter-tls") pod "node-exporter-wcbw2" (UID: "426f9a19-5c8c-4c1c-aaba-fc30325db699") : secret "node-exporter-tls" not found
Jul 23 23:14:32.240919 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.240740    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/292b6375-d147-4097-8eef-61692f076e3a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-6c8qv\" (UID: \"292b6375-d147-4097-8eef-61692f076e3a\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6c8qv"
Jul 23 23:14:32.240919 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.240766    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/426f9a19-5c8c-4c1c-aaba-fc30325db699-root\") pod \"node-exporter-wcbw2\" (UID: \"426f9a19-5c8c-4c1c-aaba-fc30325db699\") " pod="openshift-monitoring/node-exporter-wcbw2"
Jul 23 23:14:32.240919 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.240804    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sdc7\" (UniqueName: \"kubernetes.io/projected/9ee7a305-ec45-4c90-9033-05158464f844-kube-api-access-4sdc7\") pod \"kube-state-metrics-56d9f944b4-v29tt\" (UID: \"9ee7a305-ec45-4c90-9033-05158464f844\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v29tt"
Jul 23 23:14:32.240919 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.240827    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/426f9a19-5c8c-4c1c-aaba-fc30325db699-node-exporter-textfile\") pod \"node-exporter-wcbw2\" (UID: \"426f9a19-5c8c-4c1c-aaba-fc30325db699\") " pod="openshift-monitoring/node-exporter-wcbw2"
Jul 23 23:14:32.240919 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.240852    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/9ee7a305-ec45-4c90-9033-05158464f844-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-v29tt\" (UID: \"9ee7a305-ec45-4c90-9033-05158464f844\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v29tt"
Jul 23 23:14:32.240919 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.240881    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9ee7a305-ec45-4c90-9033-05158464f844-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-v29tt\" (UID: \"9ee7a305-ec45-4c90-9033-05158464f844\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v29tt"
Jul 23 23:14:32.240919 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.240911    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/426f9a19-5c8c-4c1c-aaba-fc30325db699-node-exporter-wtmp\") pod \"node-exporter-wcbw2\" (UID: \"426f9a19-5c8c-4c1c-aaba-fc30325db699\") " pod="openshift-monitoring/node-exporter-wcbw2"
Jul 23 23:14:32.241488 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.240938    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/426f9a19-5c8c-4c1c-aaba-fc30325db699-node-exporter-accelerators-collector-config\") pod \"node-exporter-wcbw2\" (UID: \"426f9a19-5c8c-4c1c-aaba-fc30325db699\") " pod="openshift-monitoring/node-exporter-wcbw2"
Jul 23 23:14:32.241488 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.240948    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9ee7a305-ec45-4c90-9033-05158464f844-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-v29tt\" (UID: \"9ee7a305-ec45-4c90-9033-05158464f844\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v29tt"
Jul 23 23:14:32.241488 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.241061    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/9ee7a305-ec45-4c90-9033-05158464f844-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-v29tt\" (UID: \"9ee7a305-ec45-4c90-9033-05158464f844\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v29tt"
Jul 23 23:14:32.241488 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.241137    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/426f9a19-5c8c-4c1c-aaba-fc30325db699-root\") pod \"node-exporter-wcbw2\" (UID: \"426f9a19-5c8c-4c1c-aaba-fc30325db699\") " pod="openshift-monitoring/node-exporter-wcbw2"
Jul 23 23:14:32.241488 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.241285    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/292b6375-d147-4097-8eef-61692f076e3a-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-6c8qv\" (UID: \"292b6375-d147-4097-8eef-61692f076e3a\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6c8qv"
Jul 23 23:14:32.241488 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.241436    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/426f9a19-5c8c-4c1c-aaba-fc30325db699-node-exporter-textfile\") pod \"node-exporter-wcbw2\" (UID: \"426f9a19-5c8c-4c1c-aaba-fc30325db699\") " pod="openshift-monitoring/node-exporter-wcbw2"
Jul 23 23:14:32.241793 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.241548    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ee7a305-ec45-4c90-9033-05158464f844-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-v29tt\" (UID: \"9ee7a305-ec45-4c90-9033-05158464f844\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v29tt"
Jul 23 23:14:32.241793 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.241572    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/426f9a19-5c8c-4c1c-aaba-fc30325db699-node-exporter-wtmp\") pod \"node-exporter-wcbw2\" (UID: \"426f9a19-5c8c-4c1c-aaba-fc30325db699\") " pod="openshift-monitoring/node-exporter-wcbw2"
Jul 23 23:14:32.241793 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:14:32.241634    2578 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 23 23:14:32.241793 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.241677    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/9ee7a305-ec45-4c90-9033-05158464f844-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-v29tt\" (UID: \"9ee7a305-ec45-4c90-9033-05158464f844\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v29tt"
Jul 23 23:14:32.241793 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:14:32.241704    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ee7a305-ec45-4c90-9033-05158464f844-kube-state-metrics-tls podName:9ee7a305-ec45-4c90-9033-05158464f844 nodeName:}" failed. No retries permitted until 2026-07-23 23:14:32.741685641 +0000 UTC m=+62.111211279 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/9ee7a305-ec45-4c90-9033-05158464f844-kube-state-metrics-tls") pod "kube-state-metrics-56d9f944b4-v29tt" (UID: "9ee7a305-ec45-4c90-9033-05158464f844") : secret "kube-state-metrics-tls" not found
Jul 23 23:14:32.242176 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.242153    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/426f9a19-5c8c-4c1c-aaba-fc30325db699-metrics-client-ca\") pod \"node-exporter-wcbw2\" (UID: \"426f9a19-5c8c-4c1c-aaba-fc30325db699\") " pod="openshift-monitoring/node-exporter-wcbw2"
Jul 23 23:14:32.243131 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.243088    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/426f9a19-5c8c-4c1c-aaba-fc30325db699-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wcbw2\" (UID: \"426f9a19-5c8c-4c1c-aaba-fc30325db699\") " pod="openshift-monitoring/node-exporter-wcbw2"
Jul 23 23:14:32.243381 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.243358    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/292b6375-d147-4097-8eef-61692f076e3a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-6c8qv\" (UID: \"292b6375-d147-4097-8eef-61692f076e3a\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6c8qv"
Jul 23 23:14:32.243836 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.243815    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/292b6375-d147-4097-8eef-61692f076e3a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-6c8qv\" (UID: \"292b6375-d147-4097-8eef-61692f076e3a\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6c8qv"
Jul 23 23:14:32.244224 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.244204    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9ee7a305-ec45-4c90-9033-05158464f844-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-v29tt\" (UID: \"9ee7a305-ec45-4c90-9033-05158464f844\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v29tt"
Jul 23 23:14:32.248352 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.248329    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8vgw\" (UniqueName: \"kubernetes.io/projected/426f9a19-5c8c-4c1c-aaba-fc30325db699-kube-api-access-s8vgw\") pod \"node-exporter-wcbw2\" (UID: \"426f9a19-5c8c-4c1c-aaba-fc30325db699\") " pod="openshift-monitoring/node-exporter-wcbw2"
Jul 23 23:14:32.251483 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.251464    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hv8q8\" (UniqueName: \"kubernetes.io/projected/292b6375-d147-4097-8eef-61692f076e3a-kube-api-access-hv8q8\") pod \"openshift-state-metrics-b8f49f5c6-6c8qv\" (UID: \"292b6375-d147-4097-8eef-61692f076e3a\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6c8qv"
Jul 23 23:14:32.254219 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.254192    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sdc7\" (UniqueName: \"kubernetes.io/projected/9ee7a305-ec45-4c90-9033-05158464f844-kube-api-access-4sdc7\") pod \"kube-state-metrics-56d9f944b4-v29tt\" (UID: \"9ee7a305-ec45-4c90-9033-05158464f844\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v29tt"
Jul 23 23:14:32.326796 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.326695    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6c8qv"
Jul 23 23:14:32.460418 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.460384    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-6c8qv"]
Jul 23 23:14:32.463569 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:14:32.463537    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod292b6375_d147_4097_8eef_61692f076e3a.slice/crio-01ebad683833df836f760e36e936a5718adc6c238fecd9d0db7187ced369accb WatchSource:0}: Error finding container 01ebad683833df836f760e36e936a5718adc6c238fecd9d0db7187ced369accb: Status 404 returned error can't find the container with id 01ebad683833df836f760e36e936a5718adc6c238fecd9d0db7187ced369accb
Jul 23 23:14:32.586253 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.586224    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6c8qv" event={"ID":"292b6375-d147-4097-8eef-61692f076e3a","Type":"ContainerStarted","Data":"2d75d639ea29ba578bb2b19303a7aca6f6537df3ff81c300e6be288bcfaf6750"}
Jul 23 23:14:32.586376 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.586279    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6c8qv" event={"ID":"292b6375-d147-4097-8eef-61692f076e3a","Type":"ContainerStarted","Data":"01ebad683833df836f760e36e936a5718adc6c238fecd9d0db7187ced369accb"}
Jul 23 23:14:32.745261 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.745220    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/426f9a19-5c8c-4c1c-aaba-fc30325db699-node-exporter-tls\") pod \"node-exporter-wcbw2\" (UID: \"426f9a19-5c8c-4c1c-aaba-fc30325db699\") " pod="openshift-monitoring/node-exporter-wcbw2"
Jul 23 23:14:32.745406 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.745305    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9ee7a305-ec45-4c90-9033-05158464f844-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-v29tt\" (UID: \"9ee7a305-ec45-4c90-9033-05158464f844\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v29tt"
Jul 23 23:14:32.747755 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.747728    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/426f9a19-5c8c-4c1c-aaba-fc30325db699-node-exporter-tls\") pod \"node-exporter-wcbw2\" (UID: \"426f9a19-5c8c-4c1c-aaba-fc30325db699\") " pod="openshift-monitoring/node-exporter-wcbw2"
Jul 23 23:14:32.747869 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.747779    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9ee7a305-ec45-4c90-9033-05158464f844-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-v29tt\" (UID: \"9ee7a305-ec45-4c90-9033-05158464f844\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v29tt"
Jul 23 23:14:32.936757 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.936724    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v29tt"
Jul 23 23:14:32.953023 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:32.952992    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wcbw2"
Jul 23 23:14:32.974136 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:14:32.974108    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod426f9a19_5c8c_4c1c_aaba_fc30325db699.slice/crio-10774c03d919009c04452eab614951642f030ac0625d38b2b50fdee75a1833ee WatchSource:0}: Error finding container 10774c03d919009c04452eab614951642f030ac0625d38b2b50fdee75a1833ee: Status 404 returned error can't find the container with id 10774c03d919009c04452eab614951642f030ac0625d38b2b50fdee75a1833ee
Jul 23 23:14:33.089214 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:33.088295    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-v29tt"]
Jul 23 23:14:33.091713 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:14:33.091682    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ee7a305_ec45_4c90_9033_05158464f844.slice/crio-030795be8ca734a7f61c58b6618865ac9546b0f08f46e3066e1005dd588c4576 WatchSource:0}: Error finding container 030795be8ca734a7f61c58b6618865ac9546b0f08f46e3066e1005dd588c4576: Status 404 returned error can't find the container with id 030795be8ca734a7f61c58b6618865ac9546b0f08f46e3066e1005dd588c4576
Jul 23 23:14:33.478893 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:33.478858    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mx6j5"
Jul 23 23:14:33.592370 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:33.592332    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wcbw2" event={"ID":"426f9a19-5c8c-4c1c-aaba-fc30325db699","Type":"ContainerStarted","Data":"10774c03d919009c04452eab614951642f030ac0625d38b2b50fdee75a1833ee"}
Jul 23 23:14:33.593857 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:33.593826    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v29tt" event={"ID":"9ee7a305-ec45-4c90-9033-05158464f844","Type":"ContainerStarted","Data":"030795be8ca734a7f61c58b6618865ac9546b0f08f46e3066e1005dd588c4576"}
Jul 23 23:14:33.596105 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:33.596077    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6c8qv" event={"ID":"292b6375-d147-4097-8eef-61692f076e3a","Type":"ContainerStarted","Data":"20ab79b53f6a560393eacb25d7691db75ae70da05082ae55c113fe84d80684cd"}
Jul 23 23:14:34.463588 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:34.463548    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-84cf6d97f9-wnz99"
Jul 23 23:14:34.464024 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:34.463617    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-84cf6d97f9-wnz99"
Jul 23 23:14:34.469352 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:34.469328    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-84cf6d97f9-wnz99"
Jul 23 23:14:34.602427 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:34.602389    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6c8qv" event={"ID":"292b6375-d147-4097-8eef-61692f076e3a","Type":"ContainerStarted","Data":"2559a1aa1ace4095531f4ee3d1c344f247af42f1911ba5d25a7a40e455e81a7b"}
Jul 23 23:14:34.604318 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:34.604289    2578 generic.go:356] "Generic (PLEG): container finished" podID="426f9a19-5c8c-4c1c-aaba-fc30325db699" containerID="667068d59c416f404bbf59f35af0c16046de936d182eab9855377118f168009b" exitCode=0
Jul 23 23:14:34.604467 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:34.604383    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wcbw2" event={"ID":"426f9a19-5c8c-4c1c-aaba-fc30325db699","Type":"ContainerDied","Data":"667068d59c416f404bbf59f35af0c16046de936d182eab9855377118f168009b"}
Jul 23 23:14:34.611142 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:34.611098    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-84cf6d97f9-wnz99"
Jul 23 23:14:34.623665 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:34.623607    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6c8qv" podStartSLOduration=2.0810388 podStartE2EDuration="3.62359342s" totalImagesPullingTime="1.54255462s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:14:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:14:34.622783946 +0000 UTC m=+63.992309605" watchObservedRunningTime="2026-07-23 23:14:34.62359342 +0000 UTC m=+63.993119080"
Jul 23 23:14:35.609736 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:35.609694    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wcbw2" event={"ID":"426f9a19-5c8c-4c1c-aaba-fc30325db699","Type":"ContainerStarted","Data":"9c5d3b29fa37e4ef92f4e8a6753d02e1927685d0ebd2e6bec57d594d1369dd45"}
Jul 23 23:14:35.610188 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:35.609770    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wcbw2" event={"ID":"426f9a19-5c8c-4c1c-aaba-fc30325db699","Type":"ContainerStarted","Data":"889ae373c52181640cb8ab6f632e3daeb0d0da07dceff058207a784c8449f6f3"}
Jul 23 23:14:35.611842 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:35.611811    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v29tt" event={"ID":"9ee7a305-ec45-4c90-9033-05158464f844","Type":"ContainerStarted","Data":"fe58ec2d5b91990db5043311df128ac076559cbf4993e88fab270a44ec9b25de"}
Jul 23 23:14:35.611960 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:35.611861    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v29tt" event={"ID":"9ee7a305-ec45-4c90-9033-05158464f844","Type":"ContainerStarted","Data":"f6844cfe061ef771f73416d579695df1549b468d724bc5aa580b230fb48b905a"}
Jul 23 23:14:35.611960 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:35.611883    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v29tt" event={"ID":"9ee7a305-ec45-4c90-9033-05158464f844","Type":"ContainerStarted","Data":"20a19d26168aeb0565baa333d70ded4910bbf6207f6df26e45869f385772bd53"}
Jul 23 23:14:35.632494 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:35.632443    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-wcbw2" podStartSLOduration=2.382417382 podStartE2EDuration="3.632428067s" totalImagesPullingTime="1.173119297s" totalInitContainerRuntime="76.891388ms" isStatefulPod=true podCreationTimestamp="2026-07-23 23:14:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:14:35.631084931 +0000 UTC m=+65.000610589" watchObservedRunningTime="2026-07-23 23:14:35.632428067 +0000 UTC m=+65.001953704"
Jul 23 23:14:35.649780 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:35.649725    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v29tt" podStartSLOduration=1.665912343 podStartE2EDuration="3.649705902s" totalImagesPullingTime="1.983793559s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:14:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:14:35.648398272 +0000 UTC m=+65.017923943" watchObservedRunningTime="2026-07-23 23:14:35.649705902 +0000 UTC m=+65.019231561"
Jul 23 23:14:35.977888 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:35.977850    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dd112f5-f0cc-4e72-917c-33e4abca79cb-metrics-certs\") pod \"network-metrics-daemon-bbv68\" (UID: \"8dd112f5-f0cc-4e72-917c-33e4abca79cb\") " pod="openshift-multus/network-metrics-daemon-bbv68"
Jul 23 23:14:35.978090 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:35.977903    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b4d53a59-83bb-4725-841c-9f5b55f85ca4-original-pull-secret\") pod \"global-pull-secret-syncer-z8d42\" (UID: \"b4d53a59-83bb-4725-841c-9f5b55f85ca4\") " pod="kube-system/global-pull-secret-syncer-z8d42"
Jul 23 23:14:35.980622 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:35.980598    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 23 23:14:35.980768 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:35.980631    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 23 23:14:35.991472 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:35.991425    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b4d53a59-83bb-4725-841c-9f5b55f85ca4-original-pull-secret\") pod \"global-pull-secret-syncer-z8d42\" (UID: \"b4d53a59-83bb-4725-841c-9f5b55f85ca4\") " pod="kube-system/global-pull-secret-syncer-z8d42"
Jul 23 23:14:35.991472 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:35.991451    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dd112f5-f0cc-4e72-917c-33e4abca79cb-metrics-certs\") pod \"network-metrics-daemon-bbv68\" (UID: \"8dd112f5-f0cc-4e72-917c-33e4abca79cb\") " pod="openshift-multus/network-metrics-daemon-bbv68"
Jul 23 23:14:36.078577 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.078536    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5bk9\" (UniqueName: \"kubernetes.io/projected/f009ab3e-edd9-4bf3-877e-37115c04d13f-kube-api-access-k5bk9\") pod \"network-check-target-5d9sj\" (UID: \"f009ab3e-edd9-4bf3-877e-37115c04d13f\") " pod="openshift-network-diagnostics/network-check-target-5d9sj"
Jul 23 23:14:36.081362 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.081342    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 23 23:14:36.092419 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.092392    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 23 23:14:36.102560 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.102534    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5bk9\" (UniqueName: \"kubernetes.io/projected/f009ab3e-edd9-4bf3-877e-37115c04d13f-kube-api-access-k5bk9\") pod \"network-check-target-5d9sj\" (UID: \"f009ab3e-edd9-4bf3-877e-37115c04d13f\") " pod="openshift-network-diagnostics/network-check-target-5d9sj"
Jul 23 23:14:36.146452 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.146420    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rl54t\""
Jul 23 23:14:36.149622 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.149608    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z8d42"
Jul 23 23:14:36.154581 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.154566    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5d9sj"
Jul 23 23:14:36.158203 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.158182    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-86k6d\""
Jul 23 23:14:36.166428 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.166407    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bbv68"
Jul 23 23:14:36.300701 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.300040    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-z8d42"]
Jul 23 23:14:36.309612 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:14:36.309575    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4d53a59_83bb_4725_841c_9f5b55f85ca4.slice/crio-32ab60dfdf4726765c06966089e8427bc68d1b5b03f9cfb61bebdf675662a2db WatchSource:0}: Error finding container 32ab60dfdf4726765c06966089e8427bc68d1b5b03f9cfb61bebdf675662a2db: Status 404 returned error can't find the container with id 32ab60dfdf4726765c06966089e8427bc68d1b5b03f9cfb61bebdf675662a2db
Jul 23 23:14:36.315190 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.315161    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5d9sj"]
Jul 23 23:14:36.333483 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.333458    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6d677d6cd8-w9t7b"]
Jul 23 23:14:36.336790 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.336769    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6d677d6cd8-w9t7b"
Jul 23 23:14:36.340353 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.340332    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 23 23:14:36.340638 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.340614    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-4zbcq\""
Jul 23 23:14:36.341903 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.341875    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 23 23:14:36.343412 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.342246    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 23 23:14:36.343412 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.342452    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-f65rjhmqhjtvc\""
Jul 23 23:14:36.343412 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.342605    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 23 23:14:36.349562 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.349537    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6d677d6cd8-w9t7b"]
Jul 23 23:14:36.357712 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.357691    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bbv68"]
Jul 23 23:14:36.360723 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:14:36.360698    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dd112f5_f0cc_4e72_917c_33e4abca79cb.slice/crio-398aa2d8353a7abb26b6edd04c9a08aad019a79e98c9583efbd1673b48f0b26f WatchSource:0}: Error finding container 398aa2d8353a7abb26b6edd04c9a08aad019a79e98c9583efbd1673b48f0b26f: Status 404 returned error can't find the container with id 398aa2d8353a7abb26b6edd04c9a08aad019a79e98c9583efbd1673b48f0b26f
Jul 23 23:14:36.481392 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.481354    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a805bdfd-024e-4561-8845-14476580ce6e-client-ca-bundle\") pod \"metrics-server-6d677d6cd8-w9t7b\" (UID: \"a805bdfd-024e-4561-8845-14476580ce6e\") " pod="openshift-monitoring/metrics-server-6d677d6cd8-w9t7b"
Jul 23 23:14:36.481533 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.481406    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a805bdfd-024e-4561-8845-14476580ce6e-secret-metrics-server-client-certs\") pod \"metrics-server-6d677d6cd8-w9t7b\" (UID: \"a805bdfd-024e-4561-8845-14476580ce6e\") " pod="openshift-monitoring/metrics-server-6d677d6cd8-w9t7b"
Jul 23 23:14:36.481533 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.481433    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a805bdfd-024e-4561-8845-14476580ce6e-metrics-server-audit-profiles\") pod \"metrics-server-6d677d6cd8-w9t7b\" (UID: \"a805bdfd-024e-4561-8845-14476580ce6e\") " pod="openshift-monitoring/metrics-server-6d677d6cd8-w9t7b"
Jul 23 23:14:36.481533 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.481465    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a805bdfd-024e-4561-8845-14476580ce6e-secret-metrics-server-tls\") pod \"metrics-server-6d677d6cd8-w9t7b\" (UID: \"a805bdfd-024e-4561-8845-14476580ce6e\") " pod="openshift-monitoring/metrics-server-6d677d6cd8-w9t7b"
Jul 23 23:14:36.481533 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.481498    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jtx6\" (UniqueName: \"kubernetes.io/projected/a805bdfd-024e-4561-8845-14476580ce6e-kube-api-access-6jtx6\") pod \"metrics-server-6d677d6cd8-w9t7b\" (UID: \"a805bdfd-024e-4561-8845-14476580ce6e\") " pod="openshift-monitoring/metrics-server-6d677d6cd8-w9t7b"
Jul 23 23:14:36.481533 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.481531    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a805bdfd-024e-4561-8845-14476580ce6e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6d677d6cd8-w9t7b\" (UID: \"a805bdfd-024e-4561-8845-14476580ce6e\") " pod="openshift-monitoring/metrics-server-6d677d6cd8-w9t7b"
Jul 23 23:14:36.481731 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.481547    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a805bdfd-024e-4561-8845-14476580ce6e-audit-log\") pod \"metrics-server-6d677d6cd8-w9t7b\" (UID: \"a805bdfd-024e-4561-8845-14476580ce6e\") " pod="openshift-monitoring/metrics-server-6d677d6cd8-w9t7b"
Jul 23 23:14:36.582439 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.582345    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jtx6\" (UniqueName: \"kubernetes.io/projected/a805bdfd-024e-4561-8845-14476580ce6e-kube-api-access-6jtx6\") pod \"metrics-server-6d677d6cd8-w9t7b\" (UID: \"a805bdfd-024e-4561-8845-14476580ce6e\") " pod="openshift-monitoring/metrics-server-6d677d6cd8-w9t7b"
Jul 23 23:14:36.582439 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.582404    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a805bdfd-024e-4561-8845-14476580ce6e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6d677d6cd8-w9t7b\" (UID: \"a805bdfd-024e-4561-8845-14476580ce6e\") " pod="openshift-monitoring/metrics-server-6d677d6cd8-w9t7b"
Jul 23 23:14:36.582439 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.582429    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a805bdfd-024e-4561-8845-14476580ce6e-audit-log\") pod \"metrics-server-6d677d6cd8-w9t7b\" (UID: \"a805bdfd-024e-4561-8845-14476580ce6e\") " pod="openshift-monitoring/metrics-server-6d677d6cd8-w9t7b"
Jul 23 23:14:36.582727 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.582452    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a805bdfd-024e-4561-8845-14476580ce6e-client-ca-bundle\") pod \"metrics-server-6d677d6cd8-w9t7b\" (UID: \"a805bdfd-024e-4561-8845-14476580ce6e\") " pod="openshift-monitoring/metrics-server-6d677d6cd8-w9t7b"
Jul 23 23:14:36.582727 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.582488    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a805bdfd-024e-4561-8845-14476580ce6e-secret-metrics-server-client-certs\") pod \"metrics-server-6d677d6cd8-w9t7b\" (UID: \"a805bdfd-024e-4561-8845-14476580ce6e\") " pod="openshift-monitoring/metrics-server-6d677d6cd8-w9t7b"
Jul 23 23:14:36.582727 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.582519    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a805bdfd-024e-4561-8845-14476580ce6e-metrics-server-audit-profiles\") pod \"metrics-server-6d677d6cd8-w9t7b\" (UID: \"a805bdfd-024e-4561-8845-14476580ce6e\") " pod="openshift-monitoring/metrics-server-6d677d6cd8-w9t7b"
Jul 23 23:14:36.582727 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.582543    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a805bdfd-024e-4561-8845-14476580ce6e-secret-metrics-server-tls\") pod \"metrics-server-6d677d6cd8-w9t7b\" (UID: \"a805bdfd-024e-4561-8845-14476580ce6e\") " pod="openshift-monitoring/metrics-server-6d677d6cd8-w9t7b"
Jul 23 23:14:36.583032 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.583000    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a805bdfd-024e-4561-8845-14476580ce6e-audit-log\") pod \"metrics-server-6d677d6cd8-w9t7b\" (UID: \"a805bdfd-024e-4561-8845-14476580ce6e\") " pod="openshift-monitoring/metrics-server-6d677d6cd8-w9t7b"
Jul 23 23:14:36.583348 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.583257    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a805bdfd-024e-4561-8845-14476580ce6e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6d677d6cd8-w9t7b\" (UID: \"a805bdfd-024e-4561-8845-14476580ce6e\") " pod="openshift-monitoring/metrics-server-6d677d6cd8-w9t7b"
Jul 23 23:14:36.583545 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.583480    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a805bdfd-024e-4561-8845-14476580ce6e-metrics-server-audit-profiles\") pod \"metrics-server-6d677d6cd8-w9t7b\" (UID: \"a805bdfd-024e-4561-8845-14476580ce6e\") " pod="openshift-monitoring/metrics-server-6d677d6cd8-w9t7b"
Jul 23 23:14:36.585205 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.585176    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a805bdfd-024e-4561-8845-14476580ce6e-secret-metrics-server-tls\") pod \"metrics-server-6d677d6cd8-w9t7b\" (UID: \"a805bdfd-024e-4561-8845-14476580ce6e\") " pod="openshift-monitoring/metrics-server-6d677d6cd8-w9t7b"
Jul 23 23:14:36.585296 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.585200    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a805bdfd-024e-4561-8845-14476580ce6e-secret-metrics-server-client-certs\") pod \"metrics-server-6d677d6cd8-w9t7b\" (UID: \"a805bdfd-024e-4561-8845-14476580ce6e\") " pod="openshift-monitoring/metrics-server-6d677d6cd8-w9t7b"
Jul 23 23:14:36.585296 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.585219    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a805bdfd-024e-4561-8845-14476580ce6e-client-ca-bundle\") pod \"metrics-server-6d677d6cd8-w9t7b\" (UID: \"a805bdfd-024e-4561-8845-14476580ce6e\") " pod="openshift-monitoring/metrics-server-6d677d6cd8-w9t7b"
Jul 23 23:14:36.591055 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.591023    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jtx6\" (UniqueName: \"kubernetes.io/projected/a805bdfd-024e-4561-8845-14476580ce6e-kube-api-access-6jtx6\") pod \"metrics-server-6d677d6cd8-w9t7b\" (UID: \"a805bdfd-024e-4561-8845-14476580ce6e\") " pod="openshift-monitoring/metrics-server-6d677d6cd8-w9t7b"
Jul 23 23:14:36.617442 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.617415    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bbv68" event={"ID":"8dd112f5-f0cc-4e72-917c-33e4abca79cb","Type":"ContainerStarted","Data":"398aa2d8353a7abb26b6edd04c9a08aad019a79e98c9583efbd1673b48f0b26f"}
Jul 23 23:14:36.618709 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.618682    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5d9sj" event={"ID":"f009ab3e-edd9-4bf3-877e-37115c04d13f","Type":"ContainerStarted","Data":"6ba227da92c397ce852d9bc99b78e7ec06d654a9e1372a251ee50dccaf07ff85"}
Jul 23 23:14:36.619813 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.619792    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-z8d42" event={"ID":"b4d53a59-83bb-4725-841c-9f5b55f85ca4","Type":"ContainerStarted","Data":"32ab60dfdf4726765c06966089e8427bc68d1b5b03f9cfb61bebdf675662a2db"}
Jul 23 23:14:36.648633 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.648606    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6d677d6cd8-w9t7b"
Jul 23 23:14:36.818546 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:36.818512    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6d677d6cd8-w9t7b"]
Jul 23 23:14:36.825490 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:14:36.825437    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda805bdfd_024e_4561_8845_14476580ce6e.slice/crio-76f58c916e4bc7d693e2d32d0b670d893dafc9c0fdc19125d1d60fb5d127e85e WatchSource:0}: Error finding container 76f58c916e4bc7d693e2d32d0b670d893dafc9c0fdc19125d1d60fb5d127e85e: Status 404 returned error can't find the container with id 76f58c916e4bc7d693e2d32d0b670d893dafc9c0fdc19125d1d60fb5d127e85e
Jul 23 23:14:37.248036 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:37.247998    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-69ccdd5d84-8kg6n"]
Jul 23 23:14:37.250783 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:37.250748    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-69ccdd5d84-8kg6n"
Jul 23 23:14:37.255956 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:37.255078    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 23 23:14:37.255956 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:37.255169    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 23 23:14:37.255956 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:37.255464    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 23 23:14:37.255956 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:37.255541    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-nn7vf\""
Jul 23 23:14:37.255956 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:37.255714    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 23 23:14:37.255956 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:37.255786    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 23 23:14:37.264499 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:37.264468    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-69ccdd5d84-8kg6n"]
Jul 23 23:14:37.265352 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:37.265334    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 23 23:14:37.389977 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:37.389924    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/795324dd-aad0-42c7-a072-45748bc9f06f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-69ccdd5d84-8kg6n\" (UID: \"795324dd-aad0-42c7-a072-45748bc9f06f\") " pod="openshift-monitoring/telemeter-client-69ccdd5d84-8kg6n"
Jul 23 23:14:37.390158 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:37.389996    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/795324dd-aad0-42c7-a072-45748bc9f06f-telemeter-client-tls\") pod \"telemeter-client-69ccdd5d84-8kg6n\" (UID: \"795324dd-aad0-42c7-a072-45748bc9f06f\") " pod="openshift-monitoring/telemeter-client-69ccdd5d84-8kg6n"
Jul 23 23:14:37.390158 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:37.390034    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/795324dd-aad0-42c7-a072-45748bc9f06f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-69ccdd5d84-8kg6n\" (UID: \"795324dd-aad0-42c7-a072-45748bc9f06f\") " pod="openshift-monitoring/telemeter-client-69ccdd5d84-8kg6n"
Jul 23 23:14:37.390158 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:37.390078    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/795324dd-aad0-42c7-a072-45748bc9f06f-secret-telemeter-client\") pod \"telemeter-client-69ccdd5d84-8kg6n\" (UID: \"795324dd-aad0-42c7-a072-45748bc9f06f\") " pod="openshift-monitoring/telemeter-client-69ccdd5d84-8kg6n"
Jul 23 23:14:37.390323 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:37.390148    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/795324dd-aad0-42c7-a072-45748bc9f06f-serving-certs-ca-bundle\") pod \"telemeter-client-69ccdd5d84-8kg6n\" (UID: \"795324dd-aad0-42c7-a072-45748bc9f06f\") " pod="openshift-monitoring/telemeter-client-69ccdd5d84-8kg6n"
Jul 23 23:14:37.390323 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:37.390198    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/795324dd-aad0-42c7-a072-45748bc9f06f-federate-client-tls\") pod \"telemeter-client-69ccdd5d84-8kg6n\" (UID: \"795324dd-aad0-42c7-a072-45748bc9f06f\") " pod="openshift-monitoring/telemeter-client-69ccdd5d84-8kg6n"
Jul 23 23:14:37.390323 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:37.390240    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/795324dd-aad0-42c7-a072-45748bc9f06f-metrics-client-ca\") pod \"telemeter-client-69ccdd5d84-8kg6n\" (UID: \"795324dd-aad0-42c7-a072-45748bc9f06f\") " pod="openshift-monitoring/telemeter-client-69ccdd5d84-8kg6n"
Jul 23 23:14:37.390323 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:37.390271    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjw9r\" (UniqueName: \"kubernetes.io/projected/795324dd-aad0-42c7-a072-45748bc9f06f-kube-api-access-rjw9r\") pod \"telemeter-client-69ccdd5d84-8kg6n\" (UID: \"795324dd-aad0-42c7-a072-45748bc9f06f\") " pod="openshift-monitoring/telemeter-client-69ccdd5d84-8kg6n"
Jul 23 23:14:37.491579 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:37.491542    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/795324dd-aad0-42c7-a072-45748bc9f06f-secret-telemeter-client\") pod \"telemeter-client-69ccdd5d84-8kg6n\" (UID: \"795324dd-aad0-42c7-a072-45748bc9f06f\") " pod="openshift-monitoring/telemeter-client-69ccdd5d84-8kg6n"
Jul 23 23:14:37.491748 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:37.491598    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/795324dd-aad0-42c7-a072-45748bc9f06f-serving-certs-ca-bundle\") pod \"telemeter-client-69ccdd5d84-8kg6n\" (UID: \"795324dd-aad0-42c7-a072-45748bc9f06f\") " pod="openshift-monitoring/telemeter-client-69ccdd5d84-8kg6n"
Jul 23 23:14:37.491748 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:37.491630    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/795324dd-aad0-42c7-a072-45748bc9f06f-federate-client-tls\") pod \"telemeter-client-69ccdd5d84-8kg6n\" (UID: \"795324dd-aad0-42c7-a072-45748bc9f06f\") " pod="openshift-monitoring/telemeter-client-69ccdd5d84-8kg6n"
Jul 23 23:14:37.491748 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:37.491685    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/795324dd-aad0-42c7-a072-45748bc9f06f-metrics-client-ca\") pod \"telemeter-client-69ccdd5d84-8kg6n\" (UID: \"795324dd-aad0-42c7-a072-45748bc9f06f\") " pod="openshift-monitoring/telemeter-client-69ccdd5d84-8kg6n"
Jul 23 23:14:37.491748 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:37.491718    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjw9r\" (UniqueName: \"kubernetes.io/projected/795324dd-aad0-42c7-a072-45748bc9f06f-kube-api-access-rjw9r\") pod \"telemeter-client-69ccdd5d84-8kg6n\" (UID: \"795324dd-aad0-42c7-a072-45748bc9f06f\") " pod="openshift-monitoring/telemeter-client-69ccdd5d84-8kg6n"
Jul 23 23:14:37.491972 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:37.491754    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/795324dd-aad0-42c7-a072-45748bc9f06f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-69ccdd5d84-8kg6n\" (UID: \"795324dd-aad0-42c7-a072-45748bc9f06f\") " pod="openshift-monitoring/telemeter-client-69ccdd5d84-8kg6n"
Jul 23 23:14:37.491972 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:37.491796    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/795324dd-aad0-42c7-a072-45748bc9f06f-telemeter-client-tls\") pod \"telemeter-client-69ccdd5d84-8kg6n\" (UID: \"795324dd-aad0-42c7-a072-45748bc9f06f\") " pod="openshift-monitoring/telemeter-client-69ccdd5d84-8kg6n"
Jul 23 23:14:37.491972 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:37.491828    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/795324dd-aad0-42c7-a072-45748bc9f06f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-69ccdd5d84-8kg6n\" (UID: \"795324dd-aad0-42c7-a072-45748bc9f06f\") " pod="openshift-monitoring/telemeter-client-69ccdd5d84-8kg6n"
Jul 23 23:14:37.492594 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:37.492374    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/795324dd-aad0-42c7-a072-45748bc9f06f-serving-certs-ca-bundle\") pod \"telemeter-client-69ccdd5d84-8kg6n\" (UID: \"795324dd-aad0-42c7-a072-45748bc9f06f\") " pod="openshift-monitoring/telemeter-client-69ccdd5d84-8kg6n"
Jul 23 23:14:37.492840 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:37.492629    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/795324dd-aad0-42c7-a072-45748bc9f06f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-69ccdd5d84-8kg6n\" (UID: \"795324dd-aad0-42c7-a072-45748bc9f06f\") " pod="openshift-monitoring/telemeter-client-69ccdd5d84-8kg6n"
Jul 23 23:14:37.492840 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:37.492765    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/795324dd-aad0-42c7-a072-45748bc9f06f-metrics-client-ca\") pod \"telemeter-client-69ccdd5d84-8kg6n\" (UID: \"795324dd-aad0-42c7-a072-45748bc9f06f\") " pod="openshift-monitoring/telemeter-client-69ccdd5d84-8kg6n"
Jul 23 23:14:37.495387 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:37.494869    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/795324dd-aad0-42c7-a072-45748bc9f06f-secret-telemeter-client\") pod \"telemeter-client-69ccdd5d84-8kg6n\" (UID: \"795324dd-aad0-42c7-a072-45748bc9f06f\") " pod="openshift-monitoring/telemeter-client-69ccdd5d84-8kg6n"
Jul 23 23:14:37.495387 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:37.495343    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/795324dd-aad0-42c7-a072-45748bc9f06f-telemeter-client-tls\") pod \"telemeter-client-69ccdd5d84-8kg6n\" (UID: \"795324dd-aad0-42c7-a072-45748bc9f06f\") " pod="openshift-monitoring/telemeter-client-69ccdd5d84-8kg6n"
Jul 23 23:14:37.496149 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:37.495960    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/795324dd-aad0-42c7-a072-45748bc9f06f-federate-client-tls\") pod \"telemeter-client-69ccdd5d84-8kg6n\" (UID: \"795324dd-aad0-42c7-a072-45748bc9f06f\") " pod="openshift-monitoring/telemeter-client-69ccdd5d84-8kg6n"
Jul 23 23:14:37.498343 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:37.498241    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/795324dd-aad0-42c7-a072-45748bc9f06f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-69ccdd5d84-8kg6n\" (UID: \"795324dd-aad0-42c7-a072-45748bc9f06f\") " pod="openshift-monitoring/telemeter-client-69ccdd5d84-8kg6n"
Jul 23 23:14:37.504316 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:37.504273    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjw9r\" (UniqueName: \"kubernetes.io/projected/795324dd-aad0-42c7-a072-45748bc9f06f-kube-api-access-rjw9r\") pod \"telemeter-client-69ccdd5d84-8kg6n\" (UID: \"795324dd-aad0-42c7-a072-45748bc9f06f\") " pod="openshift-monitoring/telemeter-client-69ccdd5d84-8kg6n"
Jul 23 23:14:37.567817 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:37.567465    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-69ccdd5d84-8kg6n"
Jul 23 23:14:37.628866 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:37.628773    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bbv68" event={"ID":"8dd112f5-f0cc-4e72-917c-33e4abca79cb","Type":"ContainerStarted","Data":"e66334ffe5143d9d970be5430a9eabfe29b1fd6ccfa50405e3b01d9deb9e3484"}
Jul 23 23:14:37.630965 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:37.630936    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6d677d6cd8-w9t7b" event={"ID":"a805bdfd-024e-4561-8845-14476580ce6e","Type":"ContainerStarted","Data":"76f58c916e4bc7d693e2d32d0b670d893dafc9c0fdc19125d1d60fb5d127e85e"}
Jul 23 23:14:37.754946 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:37.754796    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-69ccdd5d84-8kg6n"]
Jul 23 23:14:37.760803 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:14:37.760764    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod795324dd_aad0_42c7_a072_45748bc9f06f.slice/crio-69aa84a582fb04efb9edf4989699f89a965b6c84db8b2635c7612e988f3a7ed2 WatchSource:0}: Error finding container 69aa84a582fb04efb9edf4989699f89a965b6c84db8b2635c7612e988f3a7ed2: Status 404 returned error can't find the container with id 69aa84a582fb04efb9edf4989699f89a965b6c84db8b2635c7612e988f3a7ed2
Jul 23 23:14:38.384675 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.384613    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 23 23:14:38.393117 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.393080    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.397076 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.396058    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-z7sj2\""
Jul 23 23:14:38.397076 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.396627    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 23 23:14:38.399999 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.397706    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 23 23:14:38.399999 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.397985    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 23 23:14:38.399999 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.399240    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 23 23:14:38.399999 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.399335    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 23 23:14:38.399999 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.399427    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 23 23:14:38.399999 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.399677    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 23 23:14:38.399999 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.399797    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 23 23:14:38.399999 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.399872    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 23 23:14:38.400504 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.400187    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4n6g0ct4fujkf\""
Jul 23 23:14:38.400504 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.400465    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 23 23:14:38.400709 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.400689    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 23 23:14:38.405202 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.405069    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 23 23:14:38.410045 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.408768    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 23 23:14:38.410045 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.409883    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 23 23:14:38.414910 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.414885    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 23 23:14:38.502768 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.502729    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5852f69c-c825-45e2-b05e-29c7ac1d7f82-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.502950 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.502777    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5852f69c-c825-45e2-b05e-29c7ac1d7f82-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.502950 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.502806    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5852f69c-c825-45e2-b05e-29c7ac1d7f82-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.502950 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.502837    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5852f69c-c825-45e2-b05e-29c7ac1d7f82-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.502950 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.502864    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5852f69c-c825-45e2-b05e-29c7ac1d7f82-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.502950 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.502890    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5852f69c-c825-45e2-b05e-29c7ac1d7f82-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.502950 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.502913    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5852f69c-c825-45e2-b05e-29c7ac1d7f82-web-config\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.503227 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.502981    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5852f69c-c825-45e2-b05e-29c7ac1d7f82-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.503227 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.503025    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5852f69c-c825-45e2-b05e-29c7ac1d7f82-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.503227 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.503057    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5852f69c-c825-45e2-b05e-29c7ac1d7f82-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.503227 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.503081    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5852f69c-c825-45e2-b05e-29c7ac1d7f82-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.503227 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.503131    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5852f69c-c825-45e2-b05e-29c7ac1d7f82-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.503227 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.503159    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5852f69c-c825-45e2-b05e-29c7ac1d7f82-config-out\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.503227 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.503182    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5852f69c-c825-45e2-b05e-29c7ac1d7f82-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.503541 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.503244    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5852f69c-c825-45e2-b05e-29c7ac1d7f82-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.503541 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.503269    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5852f69c-c825-45e2-b05e-29c7ac1d7f82-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.503541 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.503295    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5852f69c-c825-45e2-b05e-29c7ac1d7f82-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.503541 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.503323    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4c54\" (UniqueName: \"kubernetes.io/projected/5852f69c-c825-45e2-b05e-29c7ac1d7f82-kube-api-access-m4c54\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.503541 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.503346    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5852f69c-c825-45e2-b05e-29c7ac1d7f82-config\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.503541 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.503399    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5852f69c-c825-45e2-b05e-29c7ac1d7f82-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.604266 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.604219    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5852f69c-c825-45e2-b05e-29c7ac1d7f82-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.604535 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.604271    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5852f69c-c825-45e2-b05e-29c7ac1d7f82-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.604535 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.604298    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5852f69c-c825-45e2-b05e-29c7ac1d7f82-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.604535 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.604344    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5852f69c-c825-45e2-b05e-29c7ac1d7f82-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.604535 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.604414    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5852f69c-c825-45e2-b05e-29c7ac1d7f82-config-out\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.604535 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.604439    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5852f69c-c825-45e2-b05e-29c7ac1d7f82-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.605069 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.604547    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5852f69c-c825-45e2-b05e-29c7ac1d7f82-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.605069 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.604578    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5852f69c-c825-45e2-b05e-29c7ac1d7f82-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.605069 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.604602    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5852f69c-c825-45e2-b05e-29c7ac1d7f82-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.605069 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.604626    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4c54\" (UniqueName: \"kubernetes.io/projected/5852f69c-c825-45e2-b05e-29c7ac1d7f82-kube-api-access-m4c54\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.605069 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.604667    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5852f69c-c825-45e2-b05e-29c7ac1d7f82-config\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.605069 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.604715    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5852f69c-c825-45e2-b05e-29c7ac1d7f82-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.605069 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.604742    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5852f69c-c825-45e2-b05e-29c7ac1d7f82-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.605069 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.604767    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5852f69c-c825-45e2-b05e-29c7ac1d7f82-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.605069 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.604790    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5852f69c-c825-45e2-b05e-29c7ac1d7f82-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.605069 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.604820    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5852f69c-c825-45e2-b05e-29c7ac1d7f82-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.605069 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.604847    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5852f69c-c825-45e2-b05e-29c7ac1d7f82-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.605069 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.604869    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5852f69c-c825-45e2-b05e-29c7ac1d7f82-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.605069 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.604894    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5852f69c-c825-45e2-b05e-29c7ac1d7f82-web-config\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.605069 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.604922    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5852f69c-c825-45e2-b05e-29c7ac1d7f82-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.606664 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.606569    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5852f69c-c825-45e2-b05e-29c7ac1d7f82-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.607674 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.607604    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5852f69c-c825-45e2-b05e-29c7ac1d7f82-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.608366 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.608343    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5852f69c-c825-45e2-b05e-29c7ac1d7f82-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.610881 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.609922    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5852f69c-c825-45e2-b05e-29c7ac1d7f82-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.610881 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.610415    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5852f69c-c825-45e2-b05e-29c7ac1d7f82-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.611794 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.610985    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5852f69c-c825-45e2-b05e-29c7ac1d7f82-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.612593 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.612285    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5852f69c-c825-45e2-b05e-29c7ac1d7f82-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.615730 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.615057    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5852f69c-c825-45e2-b05e-29c7ac1d7f82-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.615730 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.615433    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5852f69c-c825-45e2-b05e-29c7ac1d7f82-web-config\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.616239 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.616216    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4c54\" (UniqueName: \"kubernetes.io/projected/5852f69c-c825-45e2-b05e-29c7ac1d7f82-kube-api-access-m4c54\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.616374 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.616265    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5852f69c-c825-45e2-b05e-29c7ac1d7f82-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.619409 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.619272    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5852f69c-c825-45e2-b05e-29c7ac1d7f82-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.623082 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.621697    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5852f69c-c825-45e2-b05e-29c7ac1d7f82-config-out\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.623082 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.622146    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/5852f69c-c825-45e2-b05e-29c7ac1d7f82-config\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.623082 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.622670    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5852f69c-c825-45e2-b05e-29c7ac1d7f82-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.623289 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.623147    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5852f69c-c825-45e2-b05e-29c7ac1d7f82-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.624774 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.624285    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5852f69c-c825-45e2-b05e-29c7ac1d7f82-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.624774 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.624608    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5852f69c-c825-45e2-b05e-29c7ac1d7f82-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.626808 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.626129    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5852f69c-c825-45e2-b05e-29c7ac1d7f82-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.626808 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.626726    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5852f69c-c825-45e2-b05e-29c7ac1d7f82-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5852f69c-c825-45e2-b05e-29c7ac1d7f82\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:38.639790 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.639716    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bbv68" event={"ID":"8dd112f5-f0cc-4e72-917c-33e4abca79cb","Type":"ContainerStarted","Data":"da9a75fb8e8e005fa1506f03f6a75f3613e31d8764c76320331ff3d8016db381"}
Jul 23 23:14:38.646811 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.646120    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-69ccdd5d84-8kg6n" event={"ID":"795324dd-aad0-42c7-a072-45748bc9f06f","Type":"ContainerStarted","Data":"69aa84a582fb04efb9edf4989699f89a965b6c84db8b2635c7612e988f3a7ed2"}
Jul 23 23:14:38.663278 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.662022    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-bbv68" podStartSLOduration=65.615787949 podStartE2EDuration="1m6.66200206s" totalImagesPullingTime="1.046214111s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:13:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:14:38.660573215 +0000 UTC m=+68.030098874" watchObservedRunningTime="2026-07-23 23:14:38.66200206 +0000 UTC m=+68.031527720"
Jul 23 23:14:38.719035 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:38.718701    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:42.668311 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:42.668266    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-69ccdd5d84-8kg6n" event={"ID":"795324dd-aad0-42c7-a072-45748bc9f06f","Type":"ContainerStarted","Data":"d79d890a06c73ba99cc947e65141357a9e5da9669578ff429c702831e34e524a"}
Jul 23 23:14:42.670855 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:42.670529    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5d9sj" event={"ID":"f009ab3e-edd9-4bf3-877e-37115c04d13f","Type":"ContainerStarted","Data":"eed0e8a8e5eeea6fae130b616a91c58044f9f3139d6071b965f825415a9639ac"}
Jul 23 23:14:42.670855 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:42.670722    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-5d9sj"
Jul 23 23:14:42.672779 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:42.672691    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6d677d6cd8-w9t7b" event={"ID":"a805bdfd-024e-4561-8845-14476580ce6e","Type":"ContainerStarted","Data":"ce4cf881dfac961ca7c747f0575867055dea9a5052578f49470f6463ec177739"}
Jul 23 23:14:42.690035 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:42.689991    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-5d9sj" podStartSLOduration=64.459869104 podStartE2EDuration="1m10.689972451s" totalImagesPullingTime="6.230103347s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:13:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:14:42.68822298 +0000 UTC m=+72.057748636" watchObservedRunningTime="2026-07-23 23:14:42.689972451 +0000 UTC m=+72.059498111"
Jul 23 23:14:42.709227 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:42.709160    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 23 23:14:42.709562 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:42.709519    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6d677d6cd8-w9t7b" podStartSLOduration=0.994713069 podStartE2EDuration="6.709503402s" totalImagesPullingTime="5.714790333s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:14:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:14:42.708086402 +0000 UTC m=+72.077612061" watchObservedRunningTime="2026-07-23 23:14:42.709503402 +0000 UTC m=+72.079029062"
Jul 23 23:14:42.711812 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:14:42.711785    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5852f69c_c825_45e2_b05e_29c7ac1d7f82.slice/crio-e6f7ddca187866a217f1cf1610c5c1fc8fc41233f393ae664933c1d40e682e20 WatchSource:0}: Error finding container e6f7ddca187866a217f1cf1610c5c1fc8fc41233f393ae664933c1d40e682e20: Status 404 returned error can't find the container with id e6f7ddca187866a217f1cf1610c5c1fc8fc41233f393ae664933c1d40e682e20
Jul 23 23:14:43.678900 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:43.678853    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5852f69c-c825-45e2-b05e-29c7ac1d7f82","Type":"ContainerStarted","Data":"e6f7ddca187866a217f1cf1610c5c1fc8fc41233f393ae664933c1d40e682e20"}
Jul 23 23:14:43.680763 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:43.680722    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-z8d42" event={"ID":"b4d53a59-83bb-4725-841c-9f5b55f85ca4","Type":"ContainerStarted","Data":"8ef65edd25cb9dea37ba2ab2d38fc4146ae57754f7d7c8aaeadff8022461941c"}
Jul 23 23:14:43.696127 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:43.695943    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-z8d42" podStartSLOduration=65.444759655 podStartE2EDuration="1m11.69592322s" totalImagesPullingTime="6.251163565s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 23:13:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:14:43.695776671 +0000 UTC m=+73.065302330" watchObservedRunningTime="2026-07-23 23:14:43.69592322 +0000 UTC m=+73.065448880"
Jul 23 23:14:44.686717 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:44.686676    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-69ccdd5d84-8kg6n" event={"ID":"795324dd-aad0-42c7-a072-45748bc9f06f","Type":"ContainerStarted","Data":"5f3834be7104b429624ea1ae04d3b7a8969810b143d27c7371583db5145413ff"}
Jul 23 23:14:44.687177 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:44.686752    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-69ccdd5d84-8kg6n" event={"ID":"795324dd-aad0-42c7-a072-45748bc9f06f","Type":"ContainerStarted","Data":"70c152db6ff2c163a89d4d5fff3b075a1dd110819423473edcbf1774686bfee8"}
Jul 23 23:14:44.688216 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:44.688195    2578 generic.go:356] "Generic (PLEG): container finished" podID="5852f69c-c825-45e2-b05e-29c7ac1d7f82" containerID="7a3626420a20dfba648453cff9d09b879feaab9995ebdeaa98fdf4165d8507f5" exitCode=0
Jul 23 23:14:44.688296 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:44.688274    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5852f69c-c825-45e2-b05e-29c7ac1d7f82","Type":"ContainerDied","Data":"7a3626420a20dfba648453cff9d09b879feaab9995ebdeaa98fdf4165d8507f5"}
Jul 23 23:14:44.709134 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:44.709077    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-69ccdd5d84-8kg6n" podStartSLOduration=1.80360406 podStartE2EDuration="7.709061544s" totalImagesPullingTime="5.905457484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:14:37 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:14:44.706972347 +0000 UTC m=+74.076498043" watchObservedRunningTime="2026-07-23 23:14:44.709061544 +0000 UTC m=+74.078587203"
Jul 23 23:14:49.720364 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:49.720324    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5852f69c-c825-45e2-b05e-29c7ac1d7f82","Type":"ContainerStarted","Data":"8662e9f892f8cd2eeaec77102f259c80cb81cf09139d972cf48df70acecbae01"}
Jul 23 23:14:49.720848 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:49.720438    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5852f69c-c825-45e2-b05e-29c7ac1d7f82","Type":"ContainerStarted","Data":"d725d6db51827f3f50939e80d4074727bffccf3896b506bb54aae8104d5cbc2a"}
Jul 23 23:14:51.731956 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:51.731924    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5852f69c-c825-45e2-b05e-29c7ac1d7f82","Type":"ContainerStarted","Data":"60ee9d77e92ee7320cf6bd2a4845a45bd62f657d317ccd0ddb8fa22eb63e3639"}
Jul 23 23:14:51.732420 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:51.731992    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5852f69c-c825-45e2-b05e-29c7ac1d7f82","Type":"ContainerStarted","Data":"ab3c99413cd8398407d4c05019e592325005320e92fa9da98a16745479854ce9"}
Jul 23 23:14:51.732420 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:51.732019    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5852f69c-c825-45e2-b05e-29c7ac1d7f82","Type":"ContainerStarted","Data":"bcc7f140e356adbcc6f83f1a7c0bfa8a188c145df2301618d69455aa6e110f52"}
Jul 23 23:14:51.732420 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:51.732050    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5852f69c-c825-45e2-b05e-29c7ac1d7f82","Type":"ContainerStarted","Data":"f0e8ab34486cb2764d06b19049ece0f598a920446be00dccc9181ff68d4904a7"}
Jul 23 23:14:51.762617 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:51.762567    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=6.326065244 podStartE2EDuration="13.762549798s" totalImagesPullingTime="7.411552516s" totalInitContainerRuntime="24.932038ms" isStatefulPod=false podCreationTimestamp="2026-07-23 23:14:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:14:51.760121743 +0000 UTC m=+81.129647403" watchObservedRunningTime="2026-07-23 23:14:51.762549798 +0000 UTC m=+81.132075456"
Jul 23 23:14:53.719542 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:53.719504    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:14:56.375013 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:56.374971    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84cf6d97f9-wnz99"]
Jul 23 23:14:56.648952 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:56.648872    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6d677d6cd8-w9t7b"
Jul 23 23:14:56.648952 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:14:56.648935    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6d677d6cd8-w9t7b"
Jul 23 23:15:13.683753 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:15:13.683725    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-5d9sj"
Jul 23 23:15:16.656474 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:15:16.656371    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6d677d6cd8-w9t7b"
Jul 23 23:15:16.666725 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:15:16.666632    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6d677d6cd8-w9t7b"
Jul 23 23:15:21.394810 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:15:21.394734    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-84cf6d97f9-wnz99" podUID="c6506011-aa7d-49d6-8086-a3d097207fc0" containerName="console" containerID="cri-o://9f148f83eb6144140f26ae6a72235c05b71a537d34c19d2067ff206720a339b9" gracePeriod=15
Jul 23 23:15:21.660445 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:15:21.660371    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84cf6d97f9-wnz99"
Jul 23 23:15:21.772379 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:15:21.772336    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c6506011-aa7d-49d6-8086-a3d097207fc0-service-ca\") pod \"c6506011-aa7d-49d6-8086-a3d097207fc0\" (UID: \"c6506011-aa7d-49d6-8086-a3d097207fc0\") "
Jul 23 23:15:21.772379 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:15:21.772384    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c6506011-aa7d-49d6-8086-a3d097207fc0-oauth-serving-cert\") pod \"c6506011-aa7d-49d6-8086-a3d097207fc0\" (UID: \"c6506011-aa7d-49d6-8086-a3d097207fc0\") "
Jul 23 23:15:21.772632 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:15:21.772406    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6506011-aa7d-49d6-8086-a3d097207fc0-trusted-ca-bundle\") pod \"c6506011-aa7d-49d6-8086-a3d097207fc0\" (UID: \"c6506011-aa7d-49d6-8086-a3d097207fc0\") "
Jul 23 23:15:21.772632 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:15:21.772445    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c6506011-aa7d-49d6-8086-a3d097207fc0-console-config\") pod \"c6506011-aa7d-49d6-8086-a3d097207fc0\" (UID: \"c6506011-aa7d-49d6-8086-a3d097207fc0\") "
Jul 23 23:15:21.772632 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:15:21.772595    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c6506011-aa7d-49d6-8086-a3d097207fc0-console-oauth-config\") pod \"c6506011-aa7d-49d6-8086-a3d097207fc0\" (UID: \"c6506011-aa7d-49d6-8086-a3d097207fc0\") "
Jul 23 23:15:21.772823 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:15:21.772695    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8kcpj\" (UniqueName: \"kubernetes.io/projected/c6506011-aa7d-49d6-8086-a3d097207fc0-kube-api-access-8kcpj\") pod \"c6506011-aa7d-49d6-8086-a3d097207fc0\" (UID: \"c6506011-aa7d-49d6-8086-a3d097207fc0\") "
Jul 23 23:15:21.772823 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:15:21.772798    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c6506011-aa7d-49d6-8086-a3d097207fc0-console-serving-cert\") pod \"c6506011-aa7d-49d6-8086-a3d097207fc0\" (UID: \"c6506011-aa7d-49d6-8086-a3d097207fc0\") "
Jul 23 23:15:21.772922 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:15:21.772858    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6506011-aa7d-49d6-8086-a3d097207fc0-service-ca" (OuterVolumeSpecName: "service-ca") pod "c6506011-aa7d-49d6-8086-a3d097207fc0" (UID: "c6506011-aa7d-49d6-8086-a3d097207fc0"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:15:21.772922 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:15:21.772866    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6506011-aa7d-49d6-8086-a3d097207fc0-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c6506011-aa7d-49d6-8086-a3d097207fc0" (UID: "c6506011-aa7d-49d6-8086-a3d097207fc0"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:15:21.772922 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:15:21.772875    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6506011-aa7d-49d6-8086-a3d097207fc0-console-config" (OuterVolumeSpecName: "console-config") pod "c6506011-aa7d-49d6-8086-a3d097207fc0" (UID: "c6506011-aa7d-49d6-8086-a3d097207fc0"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:15:21.772922 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:15:21.772881    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6506011-aa7d-49d6-8086-a3d097207fc0-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c6506011-aa7d-49d6-8086-a3d097207fc0" (UID: "c6506011-aa7d-49d6-8086-a3d097207fc0"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:15:21.773108 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:15:21.773066    2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c6506011-aa7d-49d6-8086-a3d097207fc0-service-ca\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:15:21.773108 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:15:21.773088    2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c6506011-aa7d-49d6-8086-a3d097207fc0-oauth-serving-cert\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:15:21.773108 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:15:21.773103    2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6506011-aa7d-49d6-8086-a3d097207fc0-trusted-ca-bundle\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:15:21.773210 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:15:21.773116    2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c6506011-aa7d-49d6-8086-a3d097207fc0-console-config\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:15:21.775148 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:15:21.775117    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6506011-aa7d-49d6-8086-a3d097207fc0-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c6506011-aa7d-49d6-8086-a3d097207fc0" (UID: "c6506011-aa7d-49d6-8086-a3d097207fc0"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 23:15:21.775248 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:15:21.775172    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6506011-aa7d-49d6-8086-a3d097207fc0-kube-api-access-8kcpj" (OuterVolumeSpecName: "kube-api-access-8kcpj") pod "c6506011-aa7d-49d6-8086-a3d097207fc0" (UID: "c6506011-aa7d-49d6-8086-a3d097207fc0"). InnerVolumeSpecName "kube-api-access-8kcpj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:15:21.775248 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:15:21.775192    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6506011-aa7d-49d6-8086-a3d097207fc0-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c6506011-aa7d-49d6-8086-a3d097207fc0" (UID: "c6506011-aa7d-49d6-8086-a3d097207fc0"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 23:15:21.850909 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:15:21.850871    2578 generic.go:356] "Generic (PLEG): container finished" podID="c6506011-aa7d-49d6-8086-a3d097207fc0" containerID="9f148f83eb6144140f26ae6a72235c05b71a537d34c19d2067ff206720a339b9" exitCode=0
Jul 23 23:15:21.851083 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:15:21.850914    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84cf6d97f9-wnz99"
Jul 23 23:15:21.851083 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:15:21.850959    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84cf6d97f9-wnz99" event={"ID":"c6506011-aa7d-49d6-8086-a3d097207fc0","Type":"ContainerDied","Data":"9f148f83eb6144140f26ae6a72235c05b71a537d34c19d2067ff206720a339b9"}
Jul 23 23:15:21.851083 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:15:21.851013    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84cf6d97f9-wnz99" event={"ID":"c6506011-aa7d-49d6-8086-a3d097207fc0","Type":"ContainerDied","Data":"482d48404b8ae0719126970ab47446a92dc28207fc57c8c8678f9659fca3b980"}
Jul 23 23:15:21.851083 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:15:21.851034    2578 scope.go:117] "RemoveContainer" containerID="9f148f83eb6144140f26ae6a72235c05b71a537d34c19d2067ff206720a339b9"
Jul 23 23:15:21.860090 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:15:21.860065    2578 scope.go:117] "RemoveContainer" containerID="9f148f83eb6144140f26ae6a72235c05b71a537d34c19d2067ff206720a339b9"
Jul 23 23:15:21.860417 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:15:21.860391    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f148f83eb6144140f26ae6a72235c05b71a537d34c19d2067ff206720a339b9\": container with ID starting with 9f148f83eb6144140f26ae6a72235c05b71a537d34c19d2067ff206720a339b9 not found: ID does not exist" containerID="9f148f83eb6144140f26ae6a72235c05b71a537d34c19d2067ff206720a339b9"
Jul 23 23:15:21.860466 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:15:21.860428    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f148f83eb6144140f26ae6a72235c05b71a537d34c19d2067ff206720a339b9"} err="failed to get container status \"9f148f83eb6144140f26ae6a72235c05b71a537d34c19d2067ff206720a339b9\": rpc error: code = NotFound desc = could not find container \"9f148f83eb6144140f26ae6a72235c05b71a537d34c19d2067ff206720a339b9\": container with ID starting with 9f148f83eb6144140f26ae6a72235c05b71a537d34c19d2067ff206720a339b9 not found: ID does not exist"
Jul 23 23:15:21.873697 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:15:21.873634    2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c6506011-aa7d-49d6-8086-a3d097207fc0-console-oauth-config\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:15:21.873697 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:15:21.873693    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8kcpj\" (UniqueName: \"kubernetes.io/projected/c6506011-aa7d-49d6-8086-a3d097207fc0-kube-api-access-8kcpj\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:15:21.873697 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:15:21.873706    2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c6506011-aa7d-49d6-8086-a3d097207fc0-console-serving-cert\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:15:21.878527 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:15:21.878499    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84cf6d97f9-wnz99"]
Jul 23 23:15:21.882437 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:15:21.882407    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-84cf6d97f9-wnz99"]
Jul 23 23:15:23.235349 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:15:23.235318    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6506011-aa7d-49d6-8086-a3d097207fc0" path="/var/lib/kubelet/pods/c6506011-aa7d-49d6-8086-a3d097207fc0/volumes"
Jul 23 23:15:38.720092 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:15:38.720055    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:15:38.740619 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:15:38.740590    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:15:38.931028 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:15:38.931003    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 23:17:14.500020 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:14.499982    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lr24w"]
Jul 23 23:17:14.500504 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:14.500274    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6506011-aa7d-49d6-8086-a3d097207fc0" containerName="console"
Jul 23 23:17:14.500504 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:14.500285    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6506011-aa7d-49d6-8086-a3d097207fc0" containerName="console"
Jul 23 23:17:14.500504 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:14.500346    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6506011-aa7d-49d6-8086-a3d097207fc0" containerName="console"
Jul 23 23:17:14.502155 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:14.502137    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lr24w"
Jul 23 23:17:14.506677 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:14.506635    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 23 23:17:14.506806 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:14.506635    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 23 23:17:14.506806 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:14.506636    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-52j2w\""
Jul 23 23:17:14.507410 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:14.507386    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lr24w"]
Jul 23 23:17:14.633119 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:14.633081    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59gwf\" (UniqueName: \"kubernetes.io/projected/22e74d99-a1bc-4fb2-b0bf-d6371dd1b138-kube-api-access-59gwf\") pod \"openshift-lws-operator-fd8ccff4c-lr24w\" (UID: \"22e74d99-a1bc-4fb2-b0bf-d6371dd1b138\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lr24w"
Jul 23 23:17:14.633119 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:14.633124    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/22e74d99-a1bc-4fb2-b0bf-d6371dd1b138-tmp\") pod \"openshift-lws-operator-fd8ccff4c-lr24w\" (UID: \"22e74d99-a1bc-4fb2-b0bf-d6371dd1b138\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lr24w"
Jul 23 23:17:14.734421 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:14.734384    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59gwf\" (UniqueName: \"kubernetes.io/projected/22e74d99-a1bc-4fb2-b0bf-d6371dd1b138-kube-api-access-59gwf\") pod \"openshift-lws-operator-fd8ccff4c-lr24w\" (UID: \"22e74d99-a1bc-4fb2-b0bf-d6371dd1b138\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lr24w"
Jul 23 23:17:14.734421 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:14.734427    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/22e74d99-a1bc-4fb2-b0bf-d6371dd1b138-tmp\") pod \"openshift-lws-operator-fd8ccff4c-lr24w\" (UID: \"22e74d99-a1bc-4fb2-b0bf-d6371dd1b138\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lr24w"
Jul 23 23:17:14.734837 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:14.734818    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/22e74d99-a1bc-4fb2-b0bf-d6371dd1b138-tmp\") pod \"openshift-lws-operator-fd8ccff4c-lr24w\" (UID: \"22e74d99-a1bc-4fb2-b0bf-d6371dd1b138\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lr24w"
Jul 23 23:17:14.743988 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:14.743957    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59gwf\" (UniqueName: \"kubernetes.io/projected/22e74d99-a1bc-4fb2-b0bf-d6371dd1b138-kube-api-access-59gwf\") pod \"openshift-lws-operator-fd8ccff4c-lr24w\" (UID: \"22e74d99-a1bc-4fb2-b0bf-d6371dd1b138\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lr24w"
Jul 23 23:17:14.819466 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:14.819377    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lr24w"
Jul 23 23:17:14.951707 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:14.951671    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lr24w"]
Jul 23 23:17:14.954263 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:17:14.954235    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod22e74d99_a1bc_4fb2_b0bf_d6371dd1b138.slice/crio-08d2fdf2c4d4e15b0172d97edb35b42ab13399e0d8066c186f3986259a106c1c WatchSource:0}: Error finding container 08d2fdf2c4d4e15b0172d97edb35b42ab13399e0d8066c186f3986259a106c1c: Status 404 returned error can't find the container with id 08d2fdf2c4d4e15b0172d97edb35b42ab13399e0d8066c186f3986259a106c1c
Jul 23 23:17:15.272628 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:15.272591    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lr24w" event={"ID":"22e74d99-a1bc-4fb2-b0bf-d6371dd1b138","Type":"ContainerStarted","Data":"08d2fdf2c4d4e15b0172d97edb35b42ab13399e0d8066c186f3986259a106c1c"}
Jul 23 23:17:20.294234 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:20.294198    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lr24w" event={"ID":"22e74d99-a1bc-4fb2-b0bf-d6371dd1b138","Type":"ContainerStarted","Data":"3e0b18007f4c2a1607ba880b10f671b0c754d4743c854e77a92dc733f8f49e33"}
Jul 23 23:17:20.312092 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:20.312041    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lr24w" podStartSLOduration=1.9109951920000001 podStartE2EDuration="6.312025044s" totalImagesPullingTime="4.401029852s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:17:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:17:20.310376924 +0000 UTC m=+229.679902772" watchObservedRunningTime="2026-07-23 23:17:20.312025044 +0000 UTC m=+229.681550703"
Jul 23 23:17:20.708331 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:20.708297    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-hnlkv"]
Jul 23 23:17:20.710559 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:20.710541    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-hnlkv"
Jul 23 23:17:20.713208 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:20.713186    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 23 23:17:20.713308 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:20.713260    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 23 23:17:20.713308 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:20.713284    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-bktnw\""
Jul 23 23:17:20.721859 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:20.721832    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-hnlkv"]
Jul 23 23:17:20.891175 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:20.891136    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/175af0ff-e265-493f-af7f-fb085ab96f3e-bound-sa-token\") pod \"cert-manager-55b4d954d5-hnlkv\" (UID: \"175af0ff-e265-493f-af7f-fb085ab96f3e\") " pod="cert-manager/cert-manager-55b4d954d5-hnlkv"
Jul 23 23:17:20.891360 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:20.891199    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpvzb\" (UniqueName: \"kubernetes.io/projected/175af0ff-e265-493f-af7f-fb085ab96f3e-kube-api-access-bpvzb\") pod \"cert-manager-55b4d954d5-hnlkv\" (UID: \"175af0ff-e265-493f-af7f-fb085ab96f3e\") " pod="cert-manager/cert-manager-55b4d954d5-hnlkv"
Jul 23 23:17:20.991992 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:20.991904    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpvzb\" (UniqueName: \"kubernetes.io/projected/175af0ff-e265-493f-af7f-fb085ab96f3e-kube-api-access-bpvzb\") pod \"cert-manager-55b4d954d5-hnlkv\" (UID: \"175af0ff-e265-493f-af7f-fb085ab96f3e\") " pod="cert-manager/cert-manager-55b4d954d5-hnlkv"
Jul 23 23:17:20.992137 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:20.992011    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/175af0ff-e265-493f-af7f-fb085ab96f3e-bound-sa-token\") pod \"cert-manager-55b4d954d5-hnlkv\" (UID: \"175af0ff-e265-493f-af7f-fb085ab96f3e\") " pod="cert-manager/cert-manager-55b4d954d5-hnlkv"
Jul 23 23:17:21.001029 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:21.000991    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/175af0ff-e265-493f-af7f-fb085ab96f3e-bound-sa-token\") pod \"cert-manager-55b4d954d5-hnlkv\" (UID: \"175af0ff-e265-493f-af7f-fb085ab96f3e\") " pod="cert-manager/cert-manager-55b4d954d5-hnlkv"
Jul 23 23:17:21.001204 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:21.001171    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpvzb\" (UniqueName: \"kubernetes.io/projected/175af0ff-e265-493f-af7f-fb085ab96f3e-kube-api-access-bpvzb\") pod \"cert-manager-55b4d954d5-hnlkv\" (UID: \"175af0ff-e265-493f-af7f-fb085ab96f3e\") " pod="cert-manager/cert-manager-55b4d954d5-hnlkv"
Jul 23 23:17:21.020083 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:21.020042    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-hnlkv"
Jul 23 23:17:21.158826 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:21.158789    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-hnlkv"]
Jul 23 23:17:21.161794 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:17:21.161767    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod175af0ff_e265_493f_af7f_fb085ab96f3e.slice/crio-3e07f7e208ea2bcc7b219c0174219ef5a03ae1aa31e9cd93ebbdc6bb7aab5113 WatchSource:0}: Error finding container 3e07f7e208ea2bcc7b219c0174219ef5a03ae1aa31e9cd93ebbdc6bb7aab5113: Status 404 returned error can't find the container with id 3e07f7e208ea2bcc7b219c0174219ef5a03ae1aa31e9cd93ebbdc6bb7aab5113
Jul 23 23:17:21.299401 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:21.299309    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-hnlkv" event={"ID":"175af0ff-e265-493f-af7f-fb085ab96f3e","Type":"ContainerStarted","Data":"3e07f7e208ea2bcc7b219c0174219ef5a03ae1aa31e9cd93ebbdc6bb7aab5113"}
Jul 23 23:17:24.313430 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:24.313329    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-hnlkv" event={"ID":"175af0ff-e265-493f-af7f-fb085ab96f3e","Type":"ContainerStarted","Data":"c3befa7a4878b1bd85b003e4a3523faaa197a6258b71f087def3c35ef0ed37ea"}
Jul 23 23:17:24.330030 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:24.329975    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-hnlkv" podStartSLOduration=1.436512979 podStartE2EDuration="4.329960141s" totalImagesPullingTime="2.893447162s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:17:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:17:24.327389624 +0000 UTC m=+233.696915278" watchObservedRunningTime="2026-07-23 23:17:24.329960141 +0000 UTC m=+233.699485800"
Jul 23 23:17:27.093229 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:27.093189    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-68dbcfb44d-wqzsq"]
Jul 23 23:17:27.095516 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:27.095495    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-68dbcfb44d-wqzsq"
Jul 23 23:17:27.099683 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:27.099627    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 23 23:17:27.099683 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:27.099626    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 23 23:17:27.099683 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:27.099675    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-6hxf7\""
Jul 23 23:17:27.099888 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:27.099635    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 23 23:17:27.105668 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:27.105635    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-68dbcfb44d-wqzsq"]
Jul 23 23:17:27.248547 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:27.248512    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csshw\" (UniqueName: \"kubernetes.io/projected/fa8bf31f-5813-4360-9d34-8101f443652b-kube-api-access-csshw\") pod \"lws-controller-manager-68dbcfb44d-wqzsq\" (UID: \"fa8bf31f-5813-4360-9d34-8101f443652b\") " pod="openshift-lws-operator/lws-controller-manager-68dbcfb44d-wqzsq"
Jul 23 23:17:27.248737 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:27.248561    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/fa8bf31f-5813-4360-9d34-8101f443652b-metrics-cert\") pod \"lws-controller-manager-68dbcfb44d-wqzsq\" (UID: \"fa8bf31f-5813-4360-9d34-8101f443652b\") " pod="openshift-lws-operator/lws-controller-manager-68dbcfb44d-wqzsq"
Jul 23 23:17:27.248737 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:27.248596    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fa8bf31f-5813-4360-9d34-8101f443652b-cert\") pod \"lws-controller-manager-68dbcfb44d-wqzsq\" (UID: \"fa8bf31f-5813-4360-9d34-8101f443652b\") " pod="openshift-lws-operator/lws-controller-manager-68dbcfb44d-wqzsq"
Jul 23 23:17:27.248737 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:27.248645    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/fa8bf31f-5813-4360-9d34-8101f443652b-manager-config\") pod \"lws-controller-manager-68dbcfb44d-wqzsq\" (UID: \"fa8bf31f-5813-4360-9d34-8101f443652b\") " pod="openshift-lws-operator/lws-controller-manager-68dbcfb44d-wqzsq"
Jul 23 23:17:27.349436 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:27.349351    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/fa8bf31f-5813-4360-9d34-8101f443652b-metrics-cert\") pod \"lws-controller-manager-68dbcfb44d-wqzsq\" (UID: \"fa8bf31f-5813-4360-9d34-8101f443652b\") " pod="openshift-lws-operator/lws-controller-manager-68dbcfb44d-wqzsq"
Jul 23 23:17:27.349436 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:27.349402    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fa8bf31f-5813-4360-9d34-8101f443652b-cert\") pod \"lws-controller-manager-68dbcfb44d-wqzsq\" (UID: \"fa8bf31f-5813-4360-9d34-8101f443652b\") " pod="openshift-lws-operator/lws-controller-manager-68dbcfb44d-wqzsq"
Jul 23 23:17:27.349436 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:27.349436    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/fa8bf31f-5813-4360-9d34-8101f443652b-manager-config\") pod \"lws-controller-manager-68dbcfb44d-wqzsq\" (UID: \"fa8bf31f-5813-4360-9d34-8101f443652b\") " pod="openshift-lws-operator/lws-controller-manager-68dbcfb44d-wqzsq"
Jul 23 23:17:27.349758 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:27.349459    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csshw\" (UniqueName: \"kubernetes.io/projected/fa8bf31f-5813-4360-9d34-8101f443652b-kube-api-access-csshw\") pod \"lws-controller-manager-68dbcfb44d-wqzsq\" (UID: \"fa8bf31f-5813-4360-9d34-8101f443652b\") " pod="openshift-lws-operator/lws-controller-manager-68dbcfb44d-wqzsq"
Jul 23 23:17:27.350189 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:27.350159    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/fa8bf31f-5813-4360-9d34-8101f443652b-manager-config\") pod \"lws-controller-manager-68dbcfb44d-wqzsq\" (UID: \"fa8bf31f-5813-4360-9d34-8101f443652b\") " pod="openshift-lws-operator/lws-controller-manager-68dbcfb44d-wqzsq"
Jul 23 23:17:27.352066 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:27.352039    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fa8bf31f-5813-4360-9d34-8101f443652b-cert\") pod \"lws-controller-manager-68dbcfb44d-wqzsq\" (UID: \"fa8bf31f-5813-4360-9d34-8101f443652b\") " pod="openshift-lws-operator/lws-controller-manager-68dbcfb44d-wqzsq"
Jul 23 23:17:27.352066 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:27.352053    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/fa8bf31f-5813-4360-9d34-8101f443652b-metrics-cert\") pod \"lws-controller-manager-68dbcfb44d-wqzsq\" (UID: \"fa8bf31f-5813-4360-9d34-8101f443652b\") " pod="openshift-lws-operator/lws-controller-manager-68dbcfb44d-wqzsq"
Jul 23 23:17:27.366087 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:27.366049    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-csshw\" (UniqueName: \"kubernetes.io/projected/fa8bf31f-5813-4360-9d34-8101f443652b-kube-api-access-csshw\") pod \"lws-controller-manager-68dbcfb44d-wqzsq\" (UID: \"fa8bf31f-5813-4360-9d34-8101f443652b\") " pod="openshift-lws-operator/lws-controller-manager-68dbcfb44d-wqzsq"
Jul 23 23:17:27.408909 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:27.408857    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-68dbcfb44d-wqzsq"
Jul 23 23:17:27.538901 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:27.538871    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-68dbcfb44d-wqzsq"]
Jul 23 23:17:27.541148 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:17:27.541118    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa8bf31f_5813_4360_9d34_8101f443652b.slice/crio-eb1a50bf72e6a896f59cf4fcb19d6b24b34cbf3870fa06aa672e920c87f2d9d8 WatchSource:0}: Error finding container eb1a50bf72e6a896f59cf4fcb19d6b24b34cbf3870fa06aa672e920c87f2d9d8: Status 404 returned error can't find the container with id eb1a50bf72e6a896f59cf4fcb19d6b24b34cbf3870fa06aa672e920c87f2d9d8
Jul 23 23:17:28.330848 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:28.330812    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-68dbcfb44d-wqzsq" event={"ID":"fa8bf31f-5813-4360-9d34-8101f443652b","Type":"ContainerStarted","Data":"eb1a50bf72e6a896f59cf4fcb19d6b24b34cbf3870fa06aa672e920c87f2d9d8"}
Jul 23 23:17:29.337094 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:29.337055    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-68dbcfb44d-wqzsq" event={"ID":"fa8bf31f-5813-4360-9d34-8101f443652b","Type":"ContainerStarted","Data":"d422d9b163454ee92e0543c6532e9e73805588695a1d66a4416243ea3585f3e2"}
Jul 23 23:17:29.337474 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:29.337177    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-68dbcfb44d-wqzsq"
Jul 23 23:17:29.355195 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:29.355126    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-68dbcfb44d-wqzsq" podStartSLOduration=0.662623263 podStartE2EDuration="2.355107541s" totalImagesPullingTime="1.692484278s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:17:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:17:29.353999576 +0000 UTC m=+238.723525247" watchObservedRunningTime="2026-07-23 23:17:29.355107541 +0000 UTC m=+238.724633199"
Jul 23 23:17:39.991854 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:39.991820    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-mqt45"]
Jul 23 23:17:39.994640 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:39.994618    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-mqt45"
Jul 23 23:17:39.997407 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:39.997386    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 23 23:17:39.997407 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:39.997397    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-bpjbc\""
Jul 23 23:17:39.997628 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:39.997610    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 23 23:17:39.997721 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:39.997642    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 23 23:17:39.998183 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:39.998166    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 23 23:17:40.016604 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:40.016576    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-mqt45"]
Jul 23 23:17:40.041580 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:40.041549    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/67c70406-7b85-41ee-8e6d-12b6714bd46d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5fd4b8775f-mqt45\" (UID: \"67c70406-7b85-41ee-8e6d-12b6714bd46d\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-mqt45"
Jul 23 23:17:40.041754 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:40.041621    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/67c70406-7b85-41ee-8e6d-12b6714bd46d-manifests\") pod \"opendatahub-operator-controller-manager-5fd4b8775f-mqt45\" (UID: \"67c70406-7b85-41ee-8e6d-12b6714bd46d\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-mqt45"
Jul 23 23:17:40.041754 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:40.041646    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/67c70406-7b85-41ee-8e6d-12b6714bd46d-webhook-cert\") pod \"opendatahub-operator-controller-manager-5fd4b8775f-mqt45\" (UID: \"67c70406-7b85-41ee-8e6d-12b6714bd46d\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-mqt45"
Jul 23 23:17:40.041754 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:40.041685    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vb4p\" (UniqueName: \"kubernetes.io/projected/67c70406-7b85-41ee-8e6d-12b6714bd46d-kube-api-access-9vb4p\") pod \"opendatahub-operator-controller-manager-5fd4b8775f-mqt45\" (UID: \"67c70406-7b85-41ee-8e6d-12b6714bd46d\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-mqt45"
Jul 23 23:17:40.041754 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:40.041714    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/67c70406-7b85-41ee-8e6d-12b6714bd46d-tmp\") pod \"opendatahub-operator-controller-manager-5fd4b8775f-mqt45\" (UID: \"67c70406-7b85-41ee-8e6d-12b6714bd46d\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-mqt45"
Jul 23 23:17:40.142196 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:40.142160    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vb4p\" (UniqueName: \"kubernetes.io/projected/67c70406-7b85-41ee-8e6d-12b6714bd46d-kube-api-access-9vb4p\") pod \"opendatahub-operator-controller-manager-5fd4b8775f-mqt45\" (UID: \"67c70406-7b85-41ee-8e6d-12b6714bd46d\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-mqt45"
Jul 23 23:17:40.142196 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:40.142197    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/67c70406-7b85-41ee-8e6d-12b6714bd46d-tmp\") pod \"opendatahub-operator-controller-manager-5fd4b8775f-mqt45\" (UID: \"67c70406-7b85-41ee-8e6d-12b6714bd46d\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-mqt45"
Jul 23 23:17:40.142443 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:40.142248    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/67c70406-7b85-41ee-8e6d-12b6714bd46d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5fd4b8775f-mqt45\" (UID: \"67c70406-7b85-41ee-8e6d-12b6714bd46d\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-mqt45"
Jul 23 23:17:40.142443 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:40.142294    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/67c70406-7b85-41ee-8e6d-12b6714bd46d-manifests\") pod \"opendatahub-operator-controller-manager-5fd4b8775f-mqt45\" (UID: \"67c70406-7b85-41ee-8e6d-12b6714bd46d\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-mqt45"
Jul 23 23:17:40.142443 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:40.142312    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/67c70406-7b85-41ee-8e6d-12b6714bd46d-webhook-cert\") pod \"opendatahub-operator-controller-manager-5fd4b8775f-mqt45\" (UID: \"67c70406-7b85-41ee-8e6d-12b6714bd46d\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-mqt45"
Jul 23 23:17:40.142842 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:40.142809    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/67c70406-7b85-41ee-8e6d-12b6714bd46d-manifests\") pod \"opendatahub-operator-controller-manager-5fd4b8775f-mqt45\" (UID: \"67c70406-7b85-41ee-8e6d-12b6714bd46d\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-mqt45"
Jul 23 23:17:40.144962 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:40.144936    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/67c70406-7b85-41ee-8e6d-12b6714bd46d-tmp\") pod \"opendatahub-operator-controller-manager-5fd4b8775f-mqt45\" (UID: \"67c70406-7b85-41ee-8e6d-12b6714bd46d\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-mqt45"
Jul 23 23:17:40.145053 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:40.144967    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/67c70406-7b85-41ee-8e6d-12b6714bd46d-webhook-cert\") pod \"opendatahub-operator-controller-manager-5fd4b8775f-mqt45\" (UID: \"67c70406-7b85-41ee-8e6d-12b6714bd46d\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-mqt45"
Jul 23 23:17:40.145111 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:40.145096    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/67c70406-7b85-41ee-8e6d-12b6714bd46d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5fd4b8775f-mqt45\" (UID: \"67c70406-7b85-41ee-8e6d-12b6714bd46d\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-mqt45"
Jul 23 23:17:40.150563 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:40.150538    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vb4p\" (UniqueName: \"kubernetes.io/projected/67c70406-7b85-41ee-8e6d-12b6714bd46d-kube-api-access-9vb4p\") pod \"opendatahub-operator-controller-manager-5fd4b8775f-mqt45\" (UID: \"67c70406-7b85-41ee-8e6d-12b6714bd46d\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-mqt45"
Jul 23 23:17:40.304924 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:40.304820    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-mqt45"
Jul 23 23:17:40.344574 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:40.344545    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-68dbcfb44d-wqzsq"
Jul 23 23:17:40.449205 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:40.449175    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-mqt45"]
Jul 23 23:17:41.390746 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:41.390603    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-mqt45" event={"ID":"67c70406-7b85-41ee-8e6d-12b6714bd46d","Type":"ContainerStarted","Data":"9bd77a1f45ba3b209b4aab2a9bcfadd67e8238505cb7131cc67ab972f18e2425"}
Jul 23 23:17:44.404803 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:44.404767    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-mqt45" event={"ID":"67c70406-7b85-41ee-8e6d-12b6714bd46d","Type":"ContainerStarted","Data":"a06286780f20d5ad843abb5288c170ebec5da2d2760378e8de17021d5ce15a2c"}
Jul 23 23:17:45.410469 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:45.410430    2578 generic.go:356] "Generic (PLEG): container finished" podID="67c70406-7b85-41ee-8e6d-12b6714bd46d" containerID="a06286780f20d5ad843abb5288c170ebec5da2d2760378e8de17021d5ce15a2c" exitCode=0
Jul 23 23:17:45.410890 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:45.410523    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-mqt45" event={"ID":"67c70406-7b85-41ee-8e6d-12b6714bd46d","Type":"ContainerDied","Data":"a06286780f20d5ad843abb5288c170ebec5da2d2760378e8de17021d5ce15a2c"}
Jul 23 23:17:46.416883 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:46.416844    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-mqt45" event={"ID":"67c70406-7b85-41ee-8e6d-12b6714bd46d","Type":"ContainerStarted","Data":"d36ec0c24842ef5e5de305ecb57d75bcb0b6d47855a5f6cf2bc2376374cac0cb"}
Jul 23 23:17:46.417280 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:46.416951    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-mqt45"
Jul 23 23:17:46.440125 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:46.440078    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-mqt45" podStartSLOduration=2.935120916 podStartE2EDuration="7.440059317s" totalImagesPullingTime="3.489237807s" totalInitContainerRuntime="1.015700594s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:17:39 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:17:46.438105398 +0000 UTC m=+255.807631058" watchObservedRunningTime="2026-07-23 23:17:46.440059317 +0000 UTC m=+255.809584976"
Jul 23 23:17:57.426966 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:17:57.426885    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5fd4b8775f-mqt45"
Jul 23 23:18:06.676999 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:06.676966    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp"]
Jul 23 23:18:06.685685 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:06.685641    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp"
Jul 23 23:18:06.689862 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:06.689839    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-7kpjx\""
Jul 23 23:18:06.690008 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:06.689872    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 23 23:18:06.690008 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:06.689897    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 23 23:18:06.690008 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:06.689920    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 23 23:18:06.714971 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:06.714942    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp"]
Jul 23 23:18:06.782092 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:06.782052    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxhgk\" (UniqueName: \"kubernetes.io/projected/a4107bfc-1c4c-4d54-a70c-b0e65df49887-kube-api-access-pxhgk\") pod \"data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp\" (UID: \"a4107bfc-1c4c-4d54-a70c-b0e65df49887\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp"
Jul 23 23:18:06.782277 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:06.782102    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4107bfc-1c4c-4d54-a70c-b0e65df49887-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp\" (UID: \"a4107bfc-1c4c-4d54-a70c-b0e65df49887\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp"
Jul 23 23:18:06.782277 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:06.782123    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4107bfc-1c4c-4d54-a70c-b0e65df49887-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp\" (UID: \"a4107bfc-1c4c-4d54-a70c-b0e65df49887\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp"
Jul 23 23:18:06.782277 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:06.782139    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4107bfc-1c4c-4d54-a70c-b0e65df49887-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp\" (UID: \"a4107bfc-1c4c-4d54-a70c-b0e65df49887\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp"
Jul 23 23:18:06.782277 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:06.782202    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4107bfc-1c4c-4d54-a70c-b0e65df49887-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp\" (UID: \"a4107bfc-1c4c-4d54-a70c-b0e65df49887\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp"
Jul 23 23:18:06.782277 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:06.782245    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4107bfc-1c4c-4d54-a70c-b0e65df49887-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp\" (UID: \"a4107bfc-1c4c-4d54-a70c-b0e65df49887\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp"
Jul 23 23:18:06.782277 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:06.782269    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4107bfc-1c4c-4d54-a70c-b0e65df49887-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp\" (UID: \"a4107bfc-1c4c-4d54-a70c-b0e65df49887\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp"
Jul 23 23:18:06.782499 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:06.782354    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4107bfc-1c4c-4d54-a70c-b0e65df49887-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp\" (UID: \"a4107bfc-1c4c-4d54-a70c-b0e65df49887\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp"
Jul 23 23:18:06.782499 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:06.782380    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4107bfc-1c4c-4d54-a70c-b0e65df49887-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp\" (UID: \"a4107bfc-1c4c-4d54-a70c-b0e65df49887\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp"
Jul 23 23:18:06.882961 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:06.882922    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4107bfc-1c4c-4d54-a70c-b0e65df49887-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp\" (UID: \"a4107bfc-1c4c-4d54-a70c-b0e65df49887\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp"
Jul 23 23:18:06.882961 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:06.882967    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxhgk\" (UniqueName: \"kubernetes.io/projected/a4107bfc-1c4c-4d54-a70c-b0e65df49887-kube-api-access-pxhgk\") pod \"data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp\" (UID: \"a4107bfc-1c4c-4d54-a70c-b0e65df49887\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp"
Jul 23 23:18:06.883223 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:06.883004    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4107bfc-1c4c-4d54-a70c-b0e65df49887-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp\" (UID: \"a4107bfc-1c4c-4d54-a70c-b0e65df49887\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp"
Jul 23 23:18:06.883223 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:06.883024    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4107bfc-1c4c-4d54-a70c-b0e65df49887-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp\" (UID: \"a4107bfc-1c4c-4d54-a70c-b0e65df49887\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp"
Jul 23 23:18:06.883223 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:06.883041    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4107bfc-1c4c-4d54-a70c-b0e65df49887-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp\" (UID: \"a4107bfc-1c4c-4d54-a70c-b0e65df49887\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp"
Jul 23 23:18:06.883380 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:06.883293    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4107bfc-1c4c-4d54-a70c-b0e65df49887-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp\" (UID: \"a4107bfc-1c4c-4d54-a70c-b0e65df49887\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp"
Jul 23 23:18:06.883380 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:06.883337    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4107bfc-1c4c-4d54-a70c-b0e65df49887-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp\" (UID: \"a4107bfc-1c4c-4d54-a70c-b0e65df49887\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp"
Jul 23 23:18:06.883380 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:06.883372    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4107bfc-1c4c-4d54-a70c-b0e65df49887-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp\" (UID: \"a4107bfc-1c4c-4d54-a70c-b0e65df49887\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp"
Jul 23 23:18:06.883528 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:06.883385    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4107bfc-1c4c-4d54-a70c-b0e65df49887-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp\" (UID: \"a4107bfc-1c4c-4d54-a70c-b0e65df49887\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp"
Jul 23 23:18:06.883528 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:06.883436    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4107bfc-1c4c-4d54-a70c-b0e65df49887-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp\" (UID: \"a4107bfc-1c4c-4d54-a70c-b0e65df49887\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp"
Jul 23 23:18:06.883528 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:06.883479    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4107bfc-1c4c-4d54-a70c-b0e65df49887-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp\" (UID: \"a4107bfc-1c4c-4d54-a70c-b0e65df49887\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp"
Jul 23 23:18:06.883717 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:06.883632    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4107bfc-1c4c-4d54-a70c-b0e65df49887-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp\" (UID: \"a4107bfc-1c4c-4d54-a70c-b0e65df49887\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp"
Jul 23 23:18:06.883778 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:06.883754    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4107bfc-1c4c-4d54-a70c-b0e65df49887-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp\" (UID: \"a4107bfc-1c4c-4d54-a70c-b0e65df49887\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp"
Jul 23 23:18:06.883778 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:06.883766    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4107bfc-1c4c-4d54-a70c-b0e65df49887-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp\" (UID: \"a4107bfc-1c4c-4d54-a70c-b0e65df49887\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp"
Jul 23 23:18:06.885729 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:06.885704    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4107bfc-1c4c-4d54-a70c-b0e65df49887-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp\" (UID: \"a4107bfc-1c4c-4d54-a70c-b0e65df49887\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp"
Jul 23 23:18:06.885878 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:06.885860    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4107bfc-1c4c-4d54-a70c-b0e65df49887-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp\" (UID: \"a4107bfc-1c4c-4d54-a70c-b0e65df49887\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp"
Jul 23 23:18:06.892265 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:06.892231    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4107bfc-1c4c-4d54-a70c-b0e65df49887-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp\" (UID: \"a4107bfc-1c4c-4d54-a70c-b0e65df49887\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp"
Jul 23 23:18:06.892380 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:06.892360    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxhgk\" (UniqueName: \"kubernetes.io/projected/a4107bfc-1c4c-4d54-a70c-b0e65df49887-kube-api-access-pxhgk\") pod \"data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp\" (UID: \"a4107bfc-1c4c-4d54-a70c-b0e65df49887\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp"
Jul 23 23:18:06.996191 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:06.996100    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp"
Jul 23 23:18:07.136162 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:07.135989    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp"]
Jul 23 23:18:07.138471 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:18:07.138442    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4107bfc_1c4c_4d54_a70c_b0e65df49887.slice/crio-1b147f7e2bee7040f6393352f56507236b0f90abbe38e7a1a52b817a43086f59 WatchSource:0}: Error finding container 1b147f7e2bee7040f6393352f56507236b0f90abbe38e7a1a52b817a43086f59: Status 404 returned error can't find the container with id 1b147f7e2bee7040f6393352f56507236b0f90abbe38e7a1a52b817a43086f59
Jul 23 23:18:07.508425 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:07.508391    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp" event={"ID":"a4107bfc-1c4c-4d54-a70c-b0e65df49887","Type":"ContainerStarted","Data":"1b147f7e2bee7040f6393352f56507236b0f90abbe38e7a1a52b817a43086f59"}
Jul 23 23:18:07.512920 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:07.512890    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-bv462"]
Jul 23 23:18:07.517738 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:07.517716    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bv462"
Jul 23 23:18:07.520607 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:07.520583    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 23 23:18:07.520607 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:07.520596    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 23 23:18:07.520607 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:07.520607    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 23 23:18:07.520869 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:07.520706    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-b887s\""
Jul 23 23:18:07.527747 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:07.527723    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-bv462"]
Jul 23 23:18:07.591170 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:07.591134    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-bv462\" (UID: \"ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bv462"
Jul 23 23:18:07.591353 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:07.591184    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5s84k\" (UniqueName: \"kubernetes.io/projected/ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022-kube-api-access-5s84k\") pod \"istiod-openshift-gateway-5677ccb49b-bv462\" (UID: \"ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bv462"
Jul 23 23:18:07.591353 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:07.591239    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-bv462\" (UID: \"ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bv462"
Jul 23 23:18:07.591353 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:07.591301    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-bv462\" (UID: \"ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bv462"
Jul 23 23:18:07.591353 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:07.591321    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-bv462\" (UID: \"ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bv462"
Jul 23 23:18:07.591507 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:07.591384    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-bv462\" (UID: \"ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bv462"
Jul 23 23:18:07.591507 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:07.591412    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-bv462\" (UID: \"ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bv462"
Jul 23 23:18:07.692177 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:07.692139    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-bv462\" (UID: \"ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bv462"
Jul 23 23:18:07.692177 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:07.692176    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-bv462\" (UID: \"ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bv462"
Jul 23 23:18:07.692730 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:07.692203    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-bv462\" (UID: \"ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bv462"
Jul 23 23:18:07.692730 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:07.692229    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-bv462\" (UID: \"ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bv462"
Jul 23 23:18:07.692730 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:07.692279    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-bv462\" (UID: \"ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bv462"
Jul 23 23:18:07.692730 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:07.692332    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5s84k\" (UniqueName: \"kubernetes.io/projected/ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022-kube-api-access-5s84k\") pod \"istiod-openshift-gateway-5677ccb49b-bv462\" (UID: \"ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bv462"
Jul 23 23:18:07.692730 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:07.692364    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-bv462\" (UID: \"ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bv462"
Jul 23 23:18:07.693236 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:07.693200    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-bv462\" (UID: \"ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bv462"
Jul 23 23:18:07.694802 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:07.694783    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-bv462\" (UID: \"ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bv462"
Jul 23 23:18:07.694946 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:07.694916    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-bv462\" (UID: \"ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bv462"
Jul 23 23:18:07.695251 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:07.695228    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-bv462\" (UID: \"ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bv462"
Jul 23 23:18:07.695251 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:07.695244    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-bv462\" (UID: \"ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bv462"
Jul 23 23:18:07.701206 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:07.701178    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-bv462\" (UID: \"ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bv462"
Jul 23 23:18:07.701736 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:07.701712    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5s84k\" (UniqueName: \"kubernetes.io/projected/ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022-kube-api-access-5s84k\") pod \"istiod-openshift-gateway-5677ccb49b-bv462\" (UID: \"ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bv462"
Jul 23 23:18:07.828696 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:07.828578    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bv462"
Jul 23 23:18:07.988317 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:07.988029    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-bv462"]
Jul 23 23:18:07.991276 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:18:07.991245    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffb8bce6_e2dd_4d5e_8650_c0fa28f0b022.slice/crio-59e903059fbf5a273b18bdaadf4477fe641bc2c6142c56592513604ffb2805aa WatchSource:0}: Error finding container 59e903059fbf5a273b18bdaadf4477fe641bc2c6142c56592513604ffb2805aa: Status 404 returned error can't find the container with id 59e903059fbf5a273b18bdaadf4477fe641bc2c6142c56592513604ffb2805aa
Jul 23 23:18:08.515708 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:08.515645    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bv462" event={"ID":"ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022","Type":"ContainerStarted","Data":"59e903059fbf5a273b18bdaadf4477fe641bc2c6142c56592513604ffb2805aa"}
Jul 23 23:18:11.092025 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:11.091989    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:18:11.092382 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:11.092061    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:18:11.092382 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:11.092111    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:18:11.093475 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:11.093446    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:18:11.093552 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:11.093510    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:18:11.534589 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:11.534535    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bv462" event={"ID":"ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022","Type":"ContainerStarted","Data":"a48030c4c7305dea40b326dd01408fb2f4a6f20f0aa45ec3979b4ba7524c3a70"}
Jul 23 23:18:11.535312 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:11.535284    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bv462"
Jul 23 23:18:11.538059 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:11.538027    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp" event={"ID":"a4107bfc-1c4c-4d54-a70c-b0e65df49887","Type":"ContainerStarted","Data":"1c567c96bbb0bd85adced600dc485463118828b165891de6ef30994232b75bdc"}
Jul 23 23:18:11.560644 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:11.560579    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bv462" podStartSLOduration=1.4614990749999999 podStartE2EDuration="4.560562977s" totalImagesPullingTime="3.099063902s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:18:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:18:11.557629101 +0000 UTC m=+280.927154776" watchObservedRunningTime="2026-07-23 23:18:11.560562977 +0000 UTC m=+280.930088635"
Jul 23 23:18:11.580408 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:11.580335    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp" podStartSLOduration=1.62931091 podStartE2EDuration="5.580310056s" totalImagesPullingTime="3.950999146s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:18:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:18:11.579242528 +0000 UTC m=+280.948768213" watchObservedRunningTime="2026-07-23 23:18:11.580310056 +0000 UTC m=+280.949835718"
Jul 23 23:18:11.996946 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:11.996898    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp"
Jul 23 23:18:12.544022 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:12.543990    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-bv462"
Jul 23 23:18:13.001462 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:13.001432    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp"
Jul 23 23:18:13.547085 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:13.547045    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp"
Jul 23 23:18:13.547962 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:13.547943    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp"
Jul 23 23:18:31.046179 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:18:31.046151    2578 kubelet.go:1648] "Image garbage collection succeeded"
Jul 23 23:19:57.748387 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:19:57.748300    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lgm88"]
Jul 23 23:19:57.751716 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:19:57.751699    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lgm88"
Jul 23 23:19:57.754890 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:19:57.754866    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 23 23:19:57.755017 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:19:57.754995    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 23 23:19:57.755017 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:19:57.755007    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-m2lst\""
Jul 23 23:19:57.764427 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:19:57.764402    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lgm88"]
Jul 23 23:19:57.764816 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:19:57.764793    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtmtf\" (UniqueName: \"kubernetes.io/projected/2c027da3-5fa2-4e16-9ba9-a143898c66e9-kube-api-access-jtmtf\") pod \"kuadrant-operator-controller-manager-55467d9f7-lgm88\" (UID: \"2c027da3-5fa2-4e16-9ba9-a143898c66e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lgm88"
Jul 23 23:19:57.764914 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:19:57.764846    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2c027da3-5fa2-4e16-9ba9-a143898c66e9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-lgm88\" (UID: \"2c027da3-5fa2-4e16-9ba9-a143898c66e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lgm88"
Jul 23 23:19:57.852415 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:19:57.852378    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lgm88"]
Jul 23 23:19:57.852767 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:19:57.852741    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-jtmtf], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lgm88" podUID="2c027da3-5fa2-4e16-9ba9-a143898c66e9"
Jul 23 23:19:57.866093 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:19:57.866064    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtmtf\" (UniqueName: \"kubernetes.io/projected/2c027da3-5fa2-4e16-9ba9-a143898c66e9-kube-api-access-jtmtf\") pod \"kuadrant-operator-controller-manager-55467d9f7-lgm88\" (UID: \"2c027da3-5fa2-4e16-9ba9-a143898c66e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lgm88"
Jul 23 23:19:57.866239 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:19:57.866109    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2c027da3-5fa2-4e16-9ba9-a143898c66e9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-lgm88\" (UID: \"2c027da3-5fa2-4e16-9ba9-a143898c66e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lgm88"
Jul 23 23:19:57.866474 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:19:57.866458    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2c027da3-5fa2-4e16-9ba9-a143898c66e9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-lgm88\" (UID: \"2c027da3-5fa2-4e16-9ba9-a143898c66e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lgm88"
Jul 23 23:19:57.878777 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:19:57.878748    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtmtf\" (UniqueName: \"kubernetes.io/projected/2c027da3-5fa2-4e16-9ba9-a143898c66e9-kube-api-access-jtmtf\") pod \"kuadrant-operator-controller-manager-55467d9f7-lgm88\" (UID: \"2c027da3-5fa2-4e16-9ba9-a143898c66e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lgm88"
Jul 23 23:19:57.926869 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:19:57.926832    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lgm88"]
Jul 23 23:19:57.932740 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:19:57.932713    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lgm88"]
Jul 23 23:19:57.968307 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:19:57.968271    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5jq5"]
Jul 23 23:19:57.972073 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:19:57.972041    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5jq5"
Jul 23 23:19:57.988376 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:19:57.988344    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5jq5"]
Jul 23 23:19:57.996526 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:19:57.996497    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lgm88"
Jul 23 23:19:58.003298 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:19:58.003235    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lgm88"
Jul 23 23:19:58.012253 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:19:58.012216    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-lgm88\" is forbidden: User \"system:node:ip-10-0-129-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-165.ec2.internal' and this object" podUID="2c027da3-5fa2-4e16-9ba9-a143898c66e9" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lgm88"
Jul 23 23:19:58.067731 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:19:58.067696    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2c027da3-5fa2-4e16-9ba9-a143898c66e9-extensions-socket-volume\") pod \"2c027da3-5fa2-4e16-9ba9-a143898c66e9\" (UID: \"2c027da3-5fa2-4e16-9ba9-a143898c66e9\") "
Jul 23 23:19:58.067927 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:19:58.067745    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jtmtf\" (UniqueName: \"kubernetes.io/projected/2c027da3-5fa2-4e16-9ba9-a143898c66e9-kube-api-access-jtmtf\") pod \"2c027da3-5fa2-4e16-9ba9-a143898c66e9\" (UID: \"2c027da3-5fa2-4e16-9ba9-a143898c66e9\") "
Jul 23 23:19:58.067998 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:19:58.067921    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtbvl\" (UniqueName: \"kubernetes.io/projected/69344271-44d0-4123-84c3-a58c69f257f9-kube-api-access-vtbvl\") pod \"kuadrant-operator-controller-manager-8684c9ff49-g5jq5\" (UID: \"69344271-44d0-4123-84c3-a58c69f257f9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5jq5"
Jul 23 23:19:58.067998 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:19:58.067973    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/69344271-44d0-4123-84c3-a58c69f257f9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-g5jq5\" (UID: \"69344271-44d0-4123-84c3-a58c69f257f9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5jq5"
Jul 23 23:19:58.068098 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:19:58.068011    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c027da3-5fa2-4e16-9ba9-a143898c66e9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "2c027da3-5fa2-4e16-9ba9-a143898c66e9" (UID: "2c027da3-5fa2-4e16-9ba9-a143898c66e9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:19:58.068151 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:19:58.068122    2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2c027da3-5fa2-4e16-9ba9-a143898c66e9-extensions-socket-volume\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:19:58.070086 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:19:58.070059    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c027da3-5fa2-4e16-9ba9-a143898c66e9-kube-api-access-jtmtf" (OuterVolumeSpecName: "kube-api-access-jtmtf") pod "2c027da3-5fa2-4e16-9ba9-a143898c66e9" (UID: "2c027da3-5fa2-4e16-9ba9-a143898c66e9"). InnerVolumeSpecName "kube-api-access-jtmtf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:19:58.169095 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:19:58.169054    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtbvl\" (UniqueName: \"kubernetes.io/projected/69344271-44d0-4123-84c3-a58c69f257f9-kube-api-access-vtbvl\") pod \"kuadrant-operator-controller-manager-8684c9ff49-g5jq5\" (UID: \"69344271-44d0-4123-84c3-a58c69f257f9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5jq5"
Jul 23 23:19:58.169095 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:19:58.169102    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/69344271-44d0-4123-84c3-a58c69f257f9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-g5jq5\" (UID: \"69344271-44d0-4123-84c3-a58c69f257f9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5jq5"
Jul 23 23:19:58.169380 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:19:58.169185    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jtmtf\" (UniqueName: \"kubernetes.io/projected/2c027da3-5fa2-4e16-9ba9-a143898c66e9-kube-api-access-jtmtf\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:19:58.169604 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:19:58.169580    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/69344271-44d0-4123-84c3-a58c69f257f9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-g5jq5\" (UID: \"69344271-44d0-4123-84c3-a58c69f257f9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5jq5"
Jul 23 23:19:58.178546 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:19:58.178519    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtbvl\" (UniqueName: \"kubernetes.io/projected/69344271-44d0-4123-84c3-a58c69f257f9-kube-api-access-vtbvl\") pod \"kuadrant-operator-controller-manager-8684c9ff49-g5jq5\" (UID: \"69344271-44d0-4123-84c3-a58c69f257f9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5jq5"
Jul 23 23:19:58.284586 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:19:58.284489    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5jq5"
Jul 23 23:19:58.418717 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:19:58.418691    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5jq5"]
Jul 23 23:19:58.421005 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:19:58.420978    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69344271_44d0_4123_84c3_a58c69f257f9.slice/crio-02b3b43f4a3dedf495d9cbad48db71036e0e89b65157f82a3d730c164c7e5478 WatchSource:0}: Error finding container 02b3b43f4a3dedf495d9cbad48db71036e0e89b65157f82a3d730c164c7e5478: Status 404 returned error can't find the container with id 02b3b43f4a3dedf495d9cbad48db71036e0e89b65157f82a3d730c164c7e5478
Jul 23 23:19:58.423402 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:19:58.423385    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 23:19:59.002247 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:19:59.002211    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5jq5" event={"ID":"69344271-44d0-4123-84c3-a58c69f257f9","Type":"ContainerStarted","Data":"02b3b43f4a3dedf495d9cbad48db71036e0e89b65157f82a3d730c164c7e5478"}
Jul 23 23:19:59.002247 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:19:59.002240    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lgm88"
Jul 23 23:19:59.004770 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:19:59.004730    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-lgm88\" is forbidden: User \"system:node:ip-10-0-129-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-165.ec2.internal' and this object" podUID="2c027da3-5fa2-4e16-9ba9-a143898c66e9" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lgm88"
Jul 23 23:19:59.011844 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:19:59.011818    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-lgm88\" is forbidden: User \"system:node:ip-10-0-129-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-165.ec2.internal' and this object" podUID="2c027da3-5fa2-4e16-9ba9-a143898c66e9" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lgm88"
Jul 23 23:19:59.237234 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:19:59.237200    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c027da3-5fa2-4e16-9ba9-a143898c66e9" path="/var/lib/kubelet/pods/2c027da3-5fa2-4e16-9ba9-a143898c66e9/volumes"
Jul 23 23:20:04.029027 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:04.028991    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5jq5" event={"ID":"69344271-44d0-4123-84c3-a58c69f257f9","Type":"ContainerStarted","Data":"1091777daac48591e79e25fc906331af33da9b14b3d0de7df49aea5fad934335"}
Jul 23 23:20:04.029495 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:04.029069    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5jq5"
Jul 23 23:20:04.050406 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:04.050354    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5jq5" podStartSLOduration=2.359568647 podStartE2EDuration="7.050336587s" totalImagesPullingTime="4.69076794s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:19:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:20:04.048761347 +0000 UTC m=+393.418287008" watchObservedRunningTime="2026-07-23 23:20:04.050336587 +0000 UTC m=+393.419862246"
Jul 23 23:20:10.171368 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:10.171334    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5jq5"]
Jul 23 23:20:10.171834 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:10.171508    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5jq5" podUID="69344271-44d0-4123-84c3-a58c69f257f9" containerName="manager" containerID="cri-o://1091777daac48591e79e25fc906331af33da9b14b3d0de7df49aea5fad934335" gracePeriod=10
Jul 23 23:20:10.173250 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:10.173225    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5jq5"
Jul 23 23:20:10.418955 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:10.418929    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5jq5"
Jul 23 23:20:10.488666 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:10.488604    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/69344271-44d0-4123-84c3-a58c69f257f9-extensions-socket-volume\") pod \"69344271-44d0-4123-84c3-a58c69f257f9\" (UID: \"69344271-44d0-4123-84c3-a58c69f257f9\") "
Jul 23 23:20:10.488849 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:10.488762    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vtbvl\" (UniqueName: \"kubernetes.io/projected/69344271-44d0-4123-84c3-a58c69f257f9-kube-api-access-vtbvl\") pod \"69344271-44d0-4123-84c3-a58c69f257f9\" (UID: \"69344271-44d0-4123-84c3-a58c69f257f9\") "
Jul 23 23:20:10.489112 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:10.489085    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69344271-44d0-4123-84c3-a58c69f257f9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "69344271-44d0-4123-84c3-a58c69f257f9" (UID: "69344271-44d0-4123-84c3-a58c69f257f9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:20:10.491015 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:10.490983    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69344271-44d0-4123-84c3-a58c69f257f9-kube-api-access-vtbvl" (OuterVolumeSpecName: "kube-api-access-vtbvl") pod "69344271-44d0-4123-84c3-a58c69f257f9" (UID: "69344271-44d0-4123-84c3-a58c69f257f9"). InnerVolumeSpecName "kube-api-access-vtbvl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:20:10.589363 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:10.589312    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vtbvl\" (UniqueName: \"kubernetes.io/projected/69344271-44d0-4123-84c3-a58c69f257f9-kube-api-access-vtbvl\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:20:10.589363 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:10.589355    2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/69344271-44d0-4123-84c3-a58c69f257f9-extensions-socket-volume\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:20:11.059218 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:11.059180    2578 generic.go:356] "Generic (PLEG): container finished" podID="69344271-44d0-4123-84c3-a58c69f257f9" containerID="1091777daac48591e79e25fc906331af33da9b14b3d0de7df49aea5fad934335" exitCode=0
Jul 23 23:20:11.059218 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:11.059220    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5jq5"
Jul 23 23:20:11.059435 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:11.059238    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5jq5" event={"ID":"69344271-44d0-4123-84c3-a58c69f257f9","Type":"ContainerDied","Data":"1091777daac48591e79e25fc906331af33da9b14b3d0de7df49aea5fad934335"}
Jul 23 23:20:11.059435 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:11.059290    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5jq5" event={"ID":"69344271-44d0-4123-84c3-a58c69f257f9","Type":"ContainerDied","Data":"02b3b43f4a3dedf495d9cbad48db71036e0e89b65157f82a3d730c164c7e5478"}
Jul 23 23:20:11.059435 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:11.059316    2578 scope.go:117] "RemoveContainer" containerID="1091777daac48591e79e25fc906331af33da9b14b3d0de7df49aea5fad934335"
Jul 23 23:20:11.070122 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:11.070103    2578 scope.go:117] "RemoveContainer" containerID="1091777daac48591e79e25fc906331af33da9b14b3d0de7df49aea5fad934335"
Jul 23 23:20:11.070381 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:20:11.070360    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1091777daac48591e79e25fc906331af33da9b14b3d0de7df49aea5fad934335\": container with ID starting with 1091777daac48591e79e25fc906331af33da9b14b3d0de7df49aea5fad934335 not found: ID does not exist" containerID="1091777daac48591e79e25fc906331af33da9b14b3d0de7df49aea5fad934335"
Jul 23 23:20:11.070454 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:11.070389    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1091777daac48591e79e25fc906331af33da9b14b3d0de7df49aea5fad934335"} err="failed to get container status \"1091777daac48591e79e25fc906331af33da9b14b3d0de7df49aea5fad934335\": rpc error: code = NotFound desc = could not find container \"1091777daac48591e79e25fc906331af33da9b14b3d0de7df49aea5fad934335\": container with ID starting with 1091777daac48591e79e25fc906331af33da9b14b3d0de7df49aea5fad934335 not found: ID does not exist"
Jul 23 23:20:11.082040 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:11.082010    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5jq5"]
Jul 23 23:20:11.090774 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:11.090749    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g5jq5"]
Jul 23 23:20:11.236140 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:11.236106    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69344271-44d0-4123-84c3-a58c69f257f9" path="/var/lib/kubelet/pods/69344271-44d0-4123-84c3-a58c69f257f9/volumes"
Jul 23 23:20:26.572249 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:26.572211    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm"]
Jul 23 23:20:26.572729 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:26.572530    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69344271-44d0-4123-84c3-a58c69f257f9" containerName="manager"
Jul 23 23:20:26.572729 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:26.572541    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="69344271-44d0-4123-84c3-a58c69f257f9" containerName="manager"
Jul 23 23:20:26.572729 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:26.572592    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="69344271-44d0-4123-84c3-a58c69f257f9" containerName="manager"
Jul 23 23:20:26.574548 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:26.574532    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm"
Jul 23 23:20:26.577270 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:26.577250    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-vjf46\""
Jul 23 23:20:26.585884 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:26.585863    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm"]
Jul 23 23:20:26.741387 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:26.741347    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e01c2593-7e05-4f54-83db-4ac9af47dda1-workload-socket\") pod \"maas-default-gateway-openshift-default-df6776dd6-7jzmm\" (UID: \"e01c2593-7e05-4f54-83db-4ac9af47dda1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm"
Jul 23 23:20:26.741387 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:26.741390    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e01c2593-7e05-4f54-83db-4ac9af47dda1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-df6776dd6-7jzmm\" (UID: \"e01c2593-7e05-4f54-83db-4ac9af47dda1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm"
Jul 23 23:20:26.741613 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:26.741428    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e01c2593-7e05-4f54-83db-4ac9af47dda1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-df6776dd6-7jzmm\" (UID: \"e01c2593-7e05-4f54-83db-4ac9af47dda1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm"
Jul 23 23:20:26.741613 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:26.741489    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e01c2593-7e05-4f54-83db-4ac9af47dda1-istio-envoy\") pod \"maas-default-gateway-openshift-default-df6776dd6-7jzmm\" (UID: \"e01c2593-7e05-4f54-83db-4ac9af47dda1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm"
Jul 23 23:20:26.741613 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:26.741507    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e01c2593-7e05-4f54-83db-4ac9af47dda1-istio-data\") pod \"maas-default-gateway-openshift-default-df6776dd6-7jzmm\" (UID: \"e01c2593-7e05-4f54-83db-4ac9af47dda1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm"
Jul 23 23:20:26.741613 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:26.741533    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e01c2593-7e05-4f54-83db-4ac9af47dda1-credential-socket\") pod \"maas-default-gateway-openshift-default-df6776dd6-7jzmm\" (UID: \"e01c2593-7e05-4f54-83db-4ac9af47dda1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm"
Jul 23 23:20:26.741613 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:26.741557    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e01c2593-7e05-4f54-83db-4ac9af47dda1-istio-token\") pod \"maas-default-gateway-openshift-default-df6776dd6-7jzmm\" (UID: \"e01c2593-7e05-4f54-83db-4ac9af47dda1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm"
Jul 23 23:20:26.741613 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:26.741572    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xzc4\" (UniqueName: \"kubernetes.io/projected/e01c2593-7e05-4f54-83db-4ac9af47dda1-kube-api-access-2xzc4\") pod \"maas-default-gateway-openshift-default-df6776dd6-7jzmm\" (UID: \"e01c2593-7e05-4f54-83db-4ac9af47dda1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm"
Jul 23 23:20:26.741856 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:26.741634    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e01c2593-7e05-4f54-83db-4ac9af47dda1-workload-certs\") pod \"maas-default-gateway-openshift-default-df6776dd6-7jzmm\" (UID: \"e01c2593-7e05-4f54-83db-4ac9af47dda1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm"
Jul 23 23:20:26.842695 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:26.842587    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e01c2593-7e05-4f54-83db-4ac9af47dda1-workload-certs\") pod \"maas-default-gateway-openshift-default-df6776dd6-7jzmm\" (UID: \"e01c2593-7e05-4f54-83db-4ac9af47dda1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm"
Jul 23 23:20:26.842695 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:26.842678    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e01c2593-7e05-4f54-83db-4ac9af47dda1-workload-socket\") pod \"maas-default-gateway-openshift-default-df6776dd6-7jzmm\" (UID: \"e01c2593-7e05-4f54-83db-4ac9af47dda1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm"
Jul 23 23:20:26.842919 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:26.842702    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e01c2593-7e05-4f54-83db-4ac9af47dda1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-df6776dd6-7jzmm\" (UID: \"e01c2593-7e05-4f54-83db-4ac9af47dda1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm"
Jul 23 23:20:26.842919 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:26.842734    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e01c2593-7e05-4f54-83db-4ac9af47dda1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-df6776dd6-7jzmm\" (UID: \"e01c2593-7e05-4f54-83db-4ac9af47dda1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm"
Jul 23 23:20:26.842919 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:26.842764    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e01c2593-7e05-4f54-83db-4ac9af47dda1-istio-envoy\") pod \"maas-default-gateway-openshift-default-df6776dd6-7jzmm\" (UID: \"e01c2593-7e05-4f54-83db-4ac9af47dda1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm"
Jul 23 23:20:26.842919 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:26.842794    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e01c2593-7e05-4f54-83db-4ac9af47dda1-istio-data\") pod \"maas-default-gateway-openshift-default-df6776dd6-7jzmm\" (UID: \"e01c2593-7e05-4f54-83db-4ac9af47dda1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm"
Jul 23 23:20:26.842919 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:26.842830    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e01c2593-7e05-4f54-83db-4ac9af47dda1-credential-socket\") pod \"maas-default-gateway-openshift-default-df6776dd6-7jzmm\" (UID: \"e01c2593-7e05-4f54-83db-4ac9af47dda1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm"
Jul 23 23:20:26.842919 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:26.842854    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e01c2593-7e05-4f54-83db-4ac9af47dda1-istio-token\") pod \"maas-default-gateway-openshift-default-df6776dd6-7jzmm\" (UID: \"e01c2593-7e05-4f54-83db-4ac9af47dda1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm"
Jul 23 23:20:26.842919 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:26.842877    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xzc4\" (UniqueName: \"kubernetes.io/projected/e01c2593-7e05-4f54-83db-4ac9af47dda1-kube-api-access-2xzc4\") pod \"maas-default-gateway-openshift-default-df6776dd6-7jzmm\" (UID: \"e01c2593-7e05-4f54-83db-4ac9af47dda1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm"
Jul 23 23:20:26.843264 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:26.843064    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e01c2593-7e05-4f54-83db-4ac9af47dda1-workload-certs\") pod \"maas-default-gateway-openshift-default-df6776dd6-7jzmm\" (UID: \"e01c2593-7e05-4f54-83db-4ac9af47dda1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm"
Jul 23 23:20:26.843264 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:26.843176    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e01c2593-7e05-4f54-83db-4ac9af47dda1-workload-socket\") pod \"maas-default-gateway-openshift-default-df6776dd6-7jzmm\" (UID: \"e01c2593-7e05-4f54-83db-4ac9af47dda1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm"
Jul 23 23:20:26.843264 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:26.843224    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e01c2593-7e05-4f54-83db-4ac9af47dda1-credential-socket\") pod \"maas-default-gateway-openshift-default-df6776dd6-7jzmm\" (UID: \"e01c2593-7e05-4f54-83db-4ac9af47dda1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm"
Jul 23 23:20:26.843264 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:26.843238    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e01c2593-7e05-4f54-83db-4ac9af47dda1-istio-data\") pod \"maas-default-gateway-openshift-default-df6776dd6-7jzmm\" (UID: \"e01c2593-7e05-4f54-83db-4ac9af47dda1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm"
Jul 23 23:20:26.843613 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:26.843591    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e01c2593-7e05-4f54-83db-4ac9af47dda1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-df6776dd6-7jzmm\" (UID: \"e01c2593-7e05-4f54-83db-4ac9af47dda1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm"
Jul 23 23:20:26.845214 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:26.845193    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e01c2593-7e05-4f54-83db-4ac9af47dda1-istio-envoy\") pod \"maas-default-gateway-openshift-default-df6776dd6-7jzmm\" (UID: \"e01c2593-7e05-4f54-83db-4ac9af47dda1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm"
Jul 23 23:20:26.845298 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:26.845278    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e01c2593-7e05-4f54-83db-4ac9af47dda1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-df6776dd6-7jzmm\" (UID: \"e01c2593-7e05-4f54-83db-4ac9af47dda1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm"
Jul 23 23:20:26.851160 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:26.851135    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xzc4\" (UniqueName: \"kubernetes.io/projected/e01c2593-7e05-4f54-83db-4ac9af47dda1-kube-api-access-2xzc4\") pod \"maas-default-gateway-openshift-default-df6776dd6-7jzmm\" (UID: \"e01c2593-7e05-4f54-83db-4ac9af47dda1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm"
Jul 23 23:20:26.851160 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:26.851145    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e01c2593-7e05-4f54-83db-4ac9af47dda1-istio-token\") pod \"maas-default-gateway-openshift-default-df6776dd6-7jzmm\" (UID: \"e01c2593-7e05-4f54-83db-4ac9af47dda1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm"
Jul 23 23:20:26.886094 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:26.886065    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm"
Jul 23 23:20:27.028364 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:27.028339    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm"]
Jul 23 23:20:27.030534 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:20:27.030506    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode01c2593_7e05_4f54_83db_4ac9af47dda1.slice/crio-ad323b9f165af4e6d3d21e9b3070a97f49e1dda03410d057559764455c24799a WatchSource:0}: Error finding container ad323b9f165af4e6d3d21e9b3070a97f49e1dda03410d057559764455c24799a: Status 404 returned error can't find the container with id ad323b9f165af4e6d3d21e9b3070a97f49e1dda03410d057559764455c24799a
Jul 23 23:20:27.032705 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:27.032633    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:20:27.032794 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:27.032751    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:20:27.032838 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:27.032795    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:20:27.130707 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:27.130603    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm" event={"ID":"e01c2593-7e05-4f54-83db-4ac9af47dda1","Type":"ContainerStarted","Data":"0266510a03edea8089004da819428c913b55ce8409ecf6d6286d4ae319c25ec9"}
Jul 23 23:20:27.130707 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:27.130691    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm" event={"ID":"e01c2593-7e05-4f54-83db-4ac9af47dda1","Type":"ContainerStarted","Data":"ad323b9f165af4e6d3d21e9b3070a97f49e1dda03410d057559764455c24799a"}
Jul 23 23:20:27.152215 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:27.152160    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm" podStartSLOduration=1.152142661 podStartE2EDuration="1.152142661s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:20:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:20:27.151383833 +0000 UTC m=+416.520909492" watchObservedRunningTime="2026-07-23 23:20:27.152142661 +0000 UTC m=+416.521668319"
Jul 23 23:20:27.887073 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:27.887036    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm"
Jul 23 23:20:28.891160 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:28.891130    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm"
Jul 23 23:20:29.140060 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:29.140017    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm"
Jul 23 23:20:29.141101 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:20:29.141071    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm"
Jul 23 23:21:44.500117 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:44.500081    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7d567cf478-jwp69"]
Jul 23 23:21:44.503427 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:44.503411    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7d567cf478-jwp69"
Jul 23 23:21:44.506204 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:44.506185    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-6x42g\""
Jul 23 23:21:44.506295 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:44.506188    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 23 23:21:44.507288 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:44.507271    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 23:21:44.507356 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:44.507271    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 23 23:21:44.513445 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:44.513413    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7d567cf478-jwp69"]
Jul 23 23:21:44.638773 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:44.638741    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50a3bff4-f9bb-49eb-bc39-fa9206bb6472-openshift-service-ca\") pod \"maas-api-7d567cf478-jwp69\" (UID: \"50a3bff4-f9bb-49eb-bc39-fa9206bb6472\") " pod="odh-ai-gateway-infra/maas-api-7d567cf478-jwp69"
Jul 23 23:21:44.638955 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:44.638862    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltvp2\" (UniqueName: \"kubernetes.io/projected/50a3bff4-f9bb-49eb-bc39-fa9206bb6472-kube-api-access-ltvp2\") pod \"maas-api-7d567cf478-jwp69\" (UID: \"50a3bff4-f9bb-49eb-bc39-fa9206bb6472\") " pod="odh-ai-gateway-infra/maas-api-7d567cf478-jwp69"
Jul 23 23:21:44.638955 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:44.638894    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50a3bff4-f9bb-49eb-bc39-fa9206bb6472-maas-api-tls\") pod \"maas-api-7d567cf478-jwp69\" (UID: \"50a3bff4-f9bb-49eb-bc39-fa9206bb6472\") " pod="odh-ai-gateway-infra/maas-api-7d567cf478-jwp69"
Jul 23 23:21:44.739554 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:44.739520    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50a3bff4-f9bb-49eb-bc39-fa9206bb6472-maas-api-tls\") pod \"maas-api-7d567cf478-jwp69\" (UID: \"50a3bff4-f9bb-49eb-bc39-fa9206bb6472\") " pod="odh-ai-gateway-infra/maas-api-7d567cf478-jwp69"
Jul 23 23:21:44.739745 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:44.739572    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50a3bff4-f9bb-49eb-bc39-fa9206bb6472-openshift-service-ca\") pod \"maas-api-7d567cf478-jwp69\" (UID: \"50a3bff4-f9bb-49eb-bc39-fa9206bb6472\") " pod="odh-ai-gateway-infra/maas-api-7d567cf478-jwp69"
Jul 23 23:21:44.739745 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:44.739632    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltvp2\" (UniqueName: \"kubernetes.io/projected/50a3bff4-f9bb-49eb-bc39-fa9206bb6472-kube-api-access-ltvp2\") pod \"maas-api-7d567cf478-jwp69\" (UID: \"50a3bff4-f9bb-49eb-bc39-fa9206bb6472\") " pod="odh-ai-gateway-infra/maas-api-7d567cf478-jwp69"
Jul 23 23:21:44.740308 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:44.740285    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50a3bff4-f9bb-49eb-bc39-fa9206bb6472-openshift-service-ca\") pod \"maas-api-7d567cf478-jwp69\" (UID: \"50a3bff4-f9bb-49eb-bc39-fa9206bb6472\") " pod="odh-ai-gateway-infra/maas-api-7d567cf478-jwp69"
Jul 23 23:21:44.742127 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:44.742100    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50a3bff4-f9bb-49eb-bc39-fa9206bb6472-maas-api-tls\") pod \"maas-api-7d567cf478-jwp69\" (UID: \"50a3bff4-f9bb-49eb-bc39-fa9206bb6472\") " pod="odh-ai-gateway-infra/maas-api-7d567cf478-jwp69"
Jul 23 23:21:44.748919 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:44.748885    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltvp2\" (UniqueName: \"kubernetes.io/projected/50a3bff4-f9bb-49eb-bc39-fa9206bb6472-kube-api-access-ltvp2\") pod \"maas-api-7d567cf478-jwp69\" (UID: \"50a3bff4-f9bb-49eb-bc39-fa9206bb6472\") " pod="odh-ai-gateway-infra/maas-api-7d567cf478-jwp69"
Jul 23 23:21:44.814503 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:44.814413    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7d567cf478-jwp69"
Jul 23 23:21:44.952933 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:44.952910    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7d567cf478-jwp69"]
Jul 23 23:21:44.955143 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:21:44.955113    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50a3bff4_f9bb_49eb_bc39_fa9206bb6472.slice/crio-4f75b47353e51cba6fcc551ecd85fcd5215b8a47df072b279d1d455529b18c13 WatchSource:0}: Error finding container 4f75b47353e51cba6fcc551ecd85fcd5215b8a47df072b279d1d455529b18c13: Status 404 returned error can't find the container with id 4f75b47353e51cba6fcc551ecd85fcd5215b8a47df072b279d1d455529b18c13
Jul 23 23:21:45.001279 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:45.001247    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-mhd8c"]
Jul 23 23:21:45.005752 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:45.005734    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-mhd8c"
Jul 23 23:21:45.008546 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:45.008523    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-7mkbl\""
Jul 23 23:21:45.008696 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:45.008524    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 23 23:21:45.015639 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:45.015614    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-mhd8c"]
Jul 23 23:21:45.144374 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:45.144298    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpkcx\" (UniqueName: \"kubernetes.io/projected/23725880-de32-4467-b1f6-8e797fabb616-kube-api-access-cpkcx\") pod \"payload-pre-processing-7677b774bb-mhd8c\" (UID: \"23725880-de32-4467-b1f6-8e797fabb616\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-mhd8c"
Jul 23 23:21:45.144374 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:45.144353    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23725880-de32-4467-b1f6-8e797fabb616-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-mhd8c\" (UID: \"23725880-de32-4467-b1f6-8e797fabb616\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-mhd8c"
Jul 23 23:21:45.245163 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:45.245132    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpkcx\" (UniqueName: \"kubernetes.io/projected/23725880-de32-4467-b1f6-8e797fabb616-kube-api-access-cpkcx\") pod \"payload-pre-processing-7677b774bb-mhd8c\" (UID: \"23725880-de32-4467-b1f6-8e797fabb616\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-mhd8c"
Jul 23 23:21:45.245342 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:45.245191    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23725880-de32-4467-b1f6-8e797fabb616-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-mhd8c\" (UID: \"23725880-de32-4467-b1f6-8e797fabb616\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-mhd8c"
Jul 23 23:21:45.245955 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:45.245933    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23725880-de32-4467-b1f6-8e797fabb616-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-mhd8c\" (UID: \"23725880-de32-4467-b1f6-8e797fabb616\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-mhd8c"
Jul 23 23:21:45.253229 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:45.253204    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpkcx\" (UniqueName: \"kubernetes.io/projected/23725880-de32-4467-b1f6-8e797fabb616-kube-api-access-cpkcx\") pod \"payload-pre-processing-7677b774bb-mhd8c\" (UID: \"23725880-de32-4467-b1f6-8e797fabb616\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-mhd8c"
Jul 23 23:21:45.317634 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:45.317591    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-mhd8c"
Jul 23 23:21:45.445692 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:45.445610    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-mhd8c"]
Jul 23 23:21:45.447747 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:21:45.447725    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23725880_de32_4467_b1f6_8e797fabb616.slice/crio-cbda5959197234a51e49acb3c7a77d20753571330292808326836355237808d8 WatchSource:0}: Error finding container cbda5959197234a51e49acb3c7a77d20753571330292808326836355237808d8: Status 404 returned error can't find the container with id cbda5959197234a51e49acb3c7a77d20753571330292808326836355237808d8
Jul 23 23:21:45.465314 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:45.465282    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-mhd8c" event={"ID":"23725880-de32-4467-b1f6-8e797fabb616","Type":"ContainerStarted","Data":"cbda5959197234a51e49acb3c7a77d20753571330292808326836355237808d8"}
Jul 23 23:21:45.466387 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:45.466368    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7d567cf478-jwp69" event={"ID":"50a3bff4-f9bb-49eb-bc39-fa9206bb6472","Type":"ContainerStarted","Data":"4f75b47353e51cba6fcc551ecd85fcd5215b8a47df072b279d1d455529b18c13"}
Jul 23 23:21:49.513835 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:49.513794    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-mhd8c" event={"ID":"23725880-de32-4467-b1f6-8e797fabb616","Type":"ContainerStarted","Data":"9a29cb0998eaa92bb6373e7c5d555df4f0dcfbd3127a49af60f68729eb348b14"}
Jul 23 23:21:49.514339 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:49.513862    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-mhd8c"
Jul 23 23:21:49.515294 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:49.515271    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7d567cf478-jwp69" event={"ID":"50a3bff4-f9bb-49eb-bc39-fa9206bb6472","Type":"ContainerStarted","Data":"d5040ee9fc1aa8a2cb7a4f100c2d1b4ab988e49def66615eeaaf647af968b63b"}
Jul 23 23:21:49.515399 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:49.515384    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7d567cf478-jwp69"
Jul 23 23:21:49.531420 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:49.531373    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-mhd8c" podStartSLOduration=2.462160974 podStartE2EDuration="5.531359372s" totalImagesPullingTime="3.069198398s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:21:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:21:49.529642783 +0000 UTC m=+498.899168433" watchObservedRunningTime="2026-07-23 23:21:49.531359372 +0000 UTC m=+498.900885008"
Jul 23 23:21:49.545937 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:49.545894    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7d567cf478-jwp69" podStartSLOduration=1.985866023 podStartE2EDuration="5.545881001s" totalImagesPullingTime="3.560014978s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:21:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:21:49.544218192 +0000 UTC m=+498.913743851" watchObservedRunningTime="2026-07-23 23:21:49.545881001 +0000 UTC m=+498.915406659"
Jul 23 23:21:53.537440 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:53.537404    2578 generic.go:356] "Generic (PLEG): container finished" podID="e01c2593-7e05-4f54-83db-4ac9af47dda1" containerID="0266510a03edea8089004da819428c913b55ce8409ecf6d6286d4ae319c25ec9" exitCode=0
Jul 23 23:21:53.537922 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:53.537473    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm" event={"ID":"e01c2593-7e05-4f54-83db-4ac9af47dda1","Type":"ContainerDied","Data":"0266510a03edea8089004da819428c913b55ce8409ecf6d6286d4ae319c25ec9"}
Jul 23 23:21:53.537922 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:53.537869    2578 scope.go:117] "RemoveContainer" containerID="0266510a03edea8089004da819428c913b55ce8409ecf6d6286d4ae319c25ec9"
Jul 23 23:21:53.538844 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:53.538816    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:21:53.538920 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:53.538906    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:21:53.538956 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:53.538940    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:21:54.543620 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:54.543586    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm" event={"ID":"e01c2593-7e05-4f54-83db-4ac9af47dda1","Type":"ContainerStarted","Data":"99234bb605cfc60855ce3cd3c620ed0301063a9b646f00c17d434b7ae42e1cdb"}
Jul 23 23:21:54.888346 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:54.888241    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm"
Jul 23 23:21:54.888535 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:54.888393    2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-df6776dd6-7jzmm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body=
Jul 23 23:21:54.888535 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:54.888441    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm" podUID="e01c2593-7e05-4f54-83db-4ac9af47dda1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused"
Jul 23 23:21:55.525973 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:55.525946    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7d567cf478-jwp69"
Jul 23 23:21:55.887557 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:55.887442    2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-df6776dd6-7jzmm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body=
Jul 23 23:21:55.887557 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:55.887534    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm" podUID="e01c2593-7e05-4f54-83db-4ac9af47dda1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused"
Jul 23 23:21:56.886930 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:56.886879    2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-df6776dd6-7jzmm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body=
Jul 23 23:21:56.887147 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:56.886967    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm" podUID="e01c2593-7e05-4f54-83db-4ac9af47dda1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused"
Jul 23 23:21:57.887200 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:57.887148    2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-df6776dd6-7jzmm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body=
Jul 23 23:21:57.887684 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:57.887235    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm" podUID="e01c2593-7e05-4f54-83db-4ac9af47dda1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused"
Jul 23 23:21:58.890437 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:58.890404    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm"
Jul 23 23:21:58.890910 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:58.890831    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm"
Jul 23 23:21:58.891474 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:21:58.891457    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-df6776dd6-7jzmm"
Jul 23 23:22:00.522538 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:00.522506    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-mhd8c"
Jul 23 23:22:26.008010 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:26.007921    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-8697q"]
Jul 23 23:22:26.011503 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:26.011483    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8697q"
Jul 23 23:22:26.015487 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:26.015459    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 23 23:22:26.015769 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:26.015748    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-g6lpf\""
Jul 23 23:22:26.015880 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:26.015750    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 23 23:22:26.015880 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:26.015793    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 23 23:22:26.022934 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:26.022908    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-8697q"]
Jul 23 23:22:26.142563 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:26.142528    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kd8t\" (UniqueName: \"kubernetes.io/projected/48538ffa-b3c4-4866-b1cc-5d92defabff2-kube-api-access-9kd8t\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8697q\" (UID: \"48538ffa-b3c4-4866-b1cc-5d92defabff2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8697q"
Jul 23 23:22:26.142809 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:26.142604    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48538ffa-b3c4-4866-b1cc-5d92defabff2-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8697q\" (UID: \"48538ffa-b3c4-4866-b1cc-5d92defabff2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8697q"
Jul 23 23:22:26.142809 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:26.142630    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48538ffa-b3c4-4866-b1cc-5d92defabff2-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8697q\" (UID: \"48538ffa-b3c4-4866-b1cc-5d92defabff2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8697q"
Jul 23 23:22:26.142809 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:26.142688    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48538ffa-b3c4-4866-b1cc-5d92defabff2-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8697q\" (UID: \"48538ffa-b3c4-4866-b1cc-5d92defabff2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8697q"
Jul 23 23:22:26.142809 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:26.142705    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48538ffa-b3c4-4866-b1cc-5d92defabff2-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8697q\" (UID: \"48538ffa-b3c4-4866-b1cc-5d92defabff2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8697q"
Jul 23 23:22:26.142809 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:26.142745    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48538ffa-b3c4-4866-b1cc-5d92defabff2-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8697q\" (UID: \"48538ffa-b3c4-4866-b1cc-5d92defabff2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8697q"
Jul 23 23:22:26.243434 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:26.243399    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48538ffa-b3c4-4866-b1cc-5d92defabff2-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8697q\" (UID: \"48538ffa-b3c4-4866-b1cc-5d92defabff2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8697q"
Jul 23 23:22:26.243434 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:26.243435    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48538ffa-b3c4-4866-b1cc-5d92defabff2-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8697q\" (UID: \"48538ffa-b3c4-4866-b1cc-5d92defabff2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8697q"
Jul 23 23:22:26.243710 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:26.243465    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48538ffa-b3c4-4866-b1cc-5d92defabff2-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8697q\" (UID: \"48538ffa-b3c4-4866-b1cc-5d92defabff2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8697q"
Jul 23 23:22:26.243710 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:26.243480    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48538ffa-b3c4-4866-b1cc-5d92defabff2-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8697q\" (UID: \"48538ffa-b3c4-4866-b1cc-5d92defabff2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8697q"
Jul 23 23:22:26.243710 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:26.243516    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48538ffa-b3c4-4866-b1cc-5d92defabff2-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8697q\" (UID: \"48538ffa-b3c4-4866-b1cc-5d92defabff2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8697q"
Jul 23 23:22:26.243710 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:26.243546    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kd8t\" (UniqueName: \"kubernetes.io/projected/48538ffa-b3c4-4866-b1cc-5d92defabff2-kube-api-access-9kd8t\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8697q\" (UID: \"48538ffa-b3c4-4866-b1cc-5d92defabff2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8697q"
Jul 23 23:22:26.243938 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:26.243831    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48538ffa-b3c4-4866-b1cc-5d92defabff2-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8697q\" (UID: \"48538ffa-b3c4-4866-b1cc-5d92defabff2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8697q"
Jul 23 23:22:26.243938 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:26.243918    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48538ffa-b3c4-4866-b1cc-5d92defabff2-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8697q\" (UID: \"48538ffa-b3c4-4866-b1cc-5d92defabff2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8697q"
Jul 23 23:22:26.244037 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:26.244020    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48538ffa-b3c4-4866-b1cc-5d92defabff2-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8697q\" (UID: \"48538ffa-b3c4-4866-b1cc-5d92defabff2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8697q"
Jul 23 23:22:26.245881 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:26.245854    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48538ffa-b3c4-4866-b1cc-5d92defabff2-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8697q\" (UID: \"48538ffa-b3c4-4866-b1cc-5d92defabff2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8697q"
Jul 23 23:22:26.246131 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:26.246113    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48538ffa-b3c4-4866-b1cc-5d92defabff2-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8697q\" (UID: \"48538ffa-b3c4-4866-b1cc-5d92defabff2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8697q"
Jul 23 23:22:26.251933 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:26.251912    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kd8t\" (UniqueName: \"kubernetes.io/projected/48538ffa-b3c4-4866-b1cc-5d92defabff2-kube-api-access-9kd8t\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8697q\" (UID: \"48538ffa-b3c4-4866-b1cc-5d92defabff2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8697q"
Jul 23 23:22:26.323025 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:26.322911    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8697q"
Jul 23 23:22:26.453261 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:26.453220    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-8697q"]
Jul 23 23:22:26.456172 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:22:26.456143    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48538ffa_b3c4_4866_b1cc_5d92defabff2.slice/crio-c1a8930c785108318809eccfb764ade31046a7106159ffb1b838069fbeb46a1c WatchSource:0}: Error finding container c1a8930c785108318809eccfb764ade31046a7106159ffb1b838069fbeb46a1c: Status 404 returned error can't find the container with id c1a8930c785108318809eccfb764ade31046a7106159ffb1b838069fbeb46a1c
Jul 23 23:22:26.688545 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:26.688508    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8697q" event={"ID":"48538ffa-b3c4-4866-b1cc-5d92defabff2","Type":"ContainerStarted","Data":"c1a8930c785108318809eccfb764ade31046a7106159ffb1b838069fbeb46a1c"}
Jul 23 23:22:27.694317 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:27.694282    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8697q" event={"ID":"48538ffa-b3c4-4866-b1cc-5d92defabff2","Type":"ContainerStarted","Data":"a0787203a1072d97f328a6f792313f54ac918c9727c78005d984d9e60d8f8ae6"}
Jul 23 23:22:27.715174 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:27.715126    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8697q" podStartSLOduration=1.765895618 podStartE2EDuration="2.715112353s" totalImagesPullingTime="949.216735ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:22:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:22:27.714377453 +0000 UTC m=+537.083903123" watchObservedRunningTime="2026-07-23 23:22:27.715112353 +0000 UTC m=+537.084638043"
Jul 23 23:22:32.397330 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:32.397297    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-g6cjv"]
Jul 23 23:22:32.401162 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:32.401135    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6cjv"
Jul 23 23:22:32.401543 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:32.401518    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3430807-c44a-4b2a-9ce5-3f1c38a246dc-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g6cjv\" (UID: \"d3430807-c44a-4b2a-9ce5-3f1c38a246dc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6cjv"
Jul 23 23:22:32.401689 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:32.401627    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3430807-c44a-4b2a-9ce5-3f1c38a246dc-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g6cjv\" (UID: \"d3430807-c44a-4b2a-9ce5-3f1c38a246dc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6cjv"
Jul 23 23:22:32.401689 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:32.401681    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3430807-c44a-4b2a-9ce5-3f1c38a246dc-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g6cjv\" (UID: \"d3430807-c44a-4b2a-9ce5-3f1c38a246dc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6cjv"
Jul 23 23:22:32.401808 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:32.401730    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3430807-c44a-4b2a-9ce5-3f1c38a246dc-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g6cjv\" (UID: \"d3430807-c44a-4b2a-9ce5-3f1c38a246dc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6cjv"
Jul 23 23:22:32.401808 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:32.401776    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9v46q\" (UniqueName: \"kubernetes.io/projected/d3430807-c44a-4b2a-9ce5-3f1c38a246dc-kube-api-access-9v46q\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g6cjv\" (UID: \"d3430807-c44a-4b2a-9ce5-3f1c38a246dc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6cjv"
Jul 23 23:22:32.401891 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:32.401871    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3430807-c44a-4b2a-9ce5-3f1c38a246dc-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g6cjv\" (UID: \"d3430807-c44a-4b2a-9ce5-3f1c38a246dc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6cjv"
Jul 23 23:22:32.403804 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:32.403786    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 23 23:22:32.411347 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:32.411319    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-g6cjv"]
Jul 23 23:22:32.502908 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:32.502872    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v46q\" (UniqueName: \"kubernetes.io/projected/d3430807-c44a-4b2a-9ce5-3f1c38a246dc-kube-api-access-9v46q\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g6cjv\" (UID: \"d3430807-c44a-4b2a-9ce5-3f1c38a246dc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6cjv"
Jul 23 23:22:32.503084 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:32.502930    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3430807-c44a-4b2a-9ce5-3f1c38a246dc-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g6cjv\" (UID: \"d3430807-c44a-4b2a-9ce5-3f1c38a246dc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6cjv"
Jul 23 23:22:32.503084 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:32.502971    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3430807-c44a-4b2a-9ce5-3f1c38a246dc-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g6cjv\" (UID: \"d3430807-c44a-4b2a-9ce5-3f1c38a246dc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6cjv"
Jul 23 23:22:32.503084 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:32.503027    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3430807-c44a-4b2a-9ce5-3f1c38a246dc-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g6cjv\" (UID: \"d3430807-c44a-4b2a-9ce5-3f1c38a246dc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6cjv"
Jul 23 23:22:32.503084 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:32.503044    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3430807-c44a-4b2a-9ce5-3f1c38a246dc-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g6cjv\" (UID: \"d3430807-c44a-4b2a-9ce5-3f1c38a246dc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6cjv"
Jul 23 23:22:32.503084 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:32.503064    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3430807-c44a-4b2a-9ce5-3f1c38a246dc-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g6cjv\" (UID: \"d3430807-c44a-4b2a-9ce5-3f1c38a246dc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6cjv"
Jul 23 23:22:32.503411 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:32.503391    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3430807-c44a-4b2a-9ce5-3f1c38a246dc-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g6cjv\" (UID: \"d3430807-c44a-4b2a-9ce5-3f1c38a246dc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6cjv"
Jul 23 23:22:32.503501 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:32.503480    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3430807-c44a-4b2a-9ce5-3f1c38a246dc-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g6cjv\" (UID: \"d3430807-c44a-4b2a-9ce5-3f1c38a246dc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6cjv"
Jul 23 23:22:32.503552 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:32.503489    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3430807-c44a-4b2a-9ce5-3f1c38a246dc-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g6cjv\" (UID: \"d3430807-c44a-4b2a-9ce5-3f1c38a246dc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6cjv"
Jul 23 23:22:32.505399 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:32.505381    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3430807-c44a-4b2a-9ce5-3f1c38a246dc-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g6cjv\" (UID: \"d3430807-c44a-4b2a-9ce5-3f1c38a246dc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6cjv"
Jul 23 23:22:32.505588 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:32.505570    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3430807-c44a-4b2a-9ce5-3f1c38a246dc-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g6cjv\" (UID: \"d3430807-c44a-4b2a-9ce5-3f1c38a246dc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6cjv"
Jul 23 23:22:32.510959 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:32.510936    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9v46q\" (UniqueName: \"kubernetes.io/projected/d3430807-c44a-4b2a-9ce5-3f1c38a246dc-kube-api-access-9v46q\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g6cjv\" (UID: \"d3430807-c44a-4b2a-9ce5-3f1c38a246dc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6cjv"
Jul 23 23:22:32.712875 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:32.712837    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6cjv"
Jul 23 23:22:32.848132 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:32.848106    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-g6cjv"]
Jul 23 23:22:32.850122 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:22:32.850096    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3430807_c44a_4b2a_9ce5_3f1c38a246dc.slice/crio-b1fbbbfc630d89fa76e17f8ef0d7eae8382d9afbd1a221b42c7d05029fe9e18f WatchSource:0}: Error finding container b1fbbbfc630d89fa76e17f8ef0d7eae8382d9afbd1a221b42c7d05029fe9e18f: Status 404 returned error can't find the container with id b1fbbbfc630d89fa76e17f8ef0d7eae8382d9afbd1a221b42c7d05029fe9e18f
Jul 23 23:22:33.723451 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:33.723406    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6cjv" event={"ID":"d3430807-c44a-4b2a-9ce5-3f1c38a246dc","Type":"ContainerStarted","Data":"e074040080353b5818d94cbd04e89e58c39a9a5d0e914f8949f10e3d7d28f846"}
Jul 23 23:22:33.723848 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:33.723479    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6cjv" event={"ID":"d3430807-c44a-4b2a-9ce5-3f1c38a246dc","Type":"ContainerStarted","Data":"b1fbbbfc630d89fa76e17f8ef0d7eae8382d9afbd1a221b42c7d05029fe9e18f"}
Jul 23 23:22:33.747423 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:33.747369    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6cjv" podStartSLOduration=1.580146375 podStartE2EDuration="1.747352408s" totalImagesPullingTime="167.206033ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:22:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:22:33.745864865 +0000 UTC m=+543.115390534" watchObservedRunningTime="2026-07-23 23:22:33.747352408 +0000 UTC m=+543.116878066"
Jul 23 23:22:36.323455 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:36.323410    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8697q"
Jul 23 23:22:36.323986 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:36.323487    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8697q"
Jul 23 23:22:36.335895 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:36.335856    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8697q"
Jul 23 23:22:36.748249 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:36.748215    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8697q"
Jul 23 23:22:42.713674 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:42.713614    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6cjv"
Jul 23 23:22:42.714159 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:42.713702    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6cjv"
Jul 23 23:22:42.726340 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:42.726313    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6cjv"
Jul 23 23:22:42.775509 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:22:42.775477    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6cjv"
Jul 23 23:30:00.146528 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:30:00.146443    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747490-97mvp"]
Jul 23 23:30:00.149850 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:30:00.149833    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747490-97mvp"
Jul 23 23:30:00.152459 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:30:00.152438    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lzd6x\""
Jul 23 23:30:00.159029 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:30:00.159014    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747490-97mvp"
Jul 23 23:30:00.164540 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:30:00.164517    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747490-97mvp"]
Jul 23 23:30:00.291567 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:30:00.291531    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747490-97mvp"]
Jul 23 23:30:00.294158 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:30:00.294125    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc5db127_44bc_4f40_86de_b9c54d323591.slice/crio-add1d6dfc8ff90a612ebc25df92bb07720164bce85ada40dab605207e2c3975e WatchSource:0}: Error finding container add1d6dfc8ff90a612ebc25df92bb07720164bce85ada40dab605207e2c3975e: Status 404 returned error can't find the container with id add1d6dfc8ff90a612ebc25df92bb07720164bce85ada40dab605207e2c3975e
Jul 23 23:30:00.296219 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:30:00.296201    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 23:30:00.732587 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:30:00.732543    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747490-97mvp" event={"ID":"fc5db127-44bc-4f40-86de-b9c54d323591","Type":"ContainerStarted","Data":"add1d6dfc8ff90a612ebc25df92bb07720164bce85ada40dab605207e2c3975e"}
Jul 23 23:30:04.753810 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:30:04.753769    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747490-97mvp" event={"ID":"fc5db127-44bc-4f40-86de-b9c54d323591","Type":"ContainerStarted","Data":"ab358d9fc1728b0bf92388e817140830c14e6bab49f5e93140108badd02b3338"}
Jul 23 23:30:04.771311 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:30:04.771253    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747490-97mvp" podStartSLOduration=1.012664982 podStartE2EDuration="4.771235042s" totalImagesPullingTime="3.75857006s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:30:04.770503053 +0000 UTC m=+994.140028712" watchObservedRunningTime="2026-07-23 23:30:04.771235042 +0000 UTC m=+994.140760703"
Jul 23 23:30:05.759971 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:30:05.759923    2578 generic.go:356] "Generic (PLEG): container finished" podID="fc5db127-44bc-4f40-86de-b9c54d323591" containerID="ab358d9fc1728b0bf92388e817140830c14e6bab49f5e93140108badd02b3338" exitCode=0
Jul 23 23:30:05.760453 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:30:05.760014    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747490-97mvp" event={"ID":"fc5db127-44bc-4f40-86de-b9c54d323591","Type":"ContainerDied","Data":"ab358d9fc1728b0bf92388e817140830c14e6bab49f5e93140108badd02b3338"}
Jul 23 23:30:06.885771 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:30:06.885742    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747490-97mvp"
Jul 23 23:30:07.770338 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:30:07.770312    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747490-97mvp"
Jul 23 23:30:07.770506 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:30:07.770336    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747490-97mvp" event={"ID":"fc5db127-44bc-4f40-86de-b9c54d323591","Type":"ContainerDied","Data":"add1d6dfc8ff90a612ebc25df92bb07720164bce85ada40dab605207e2c3975e"}
Jul 23 23:30:07.770506 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:30:07.770384    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="add1d6dfc8ff90a612ebc25df92bb07720164bce85ada40dab605207e2c3975e"
Jul 23 23:38:34.237669 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:34.237624    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4"]
Jul 23 23:38:34.238139 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:34.238017    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc5db127-44bc-4f40-86de-b9c54d323591" containerName="cleanup"
Jul 23 23:38:34.238139 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:34.238030    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc5db127-44bc-4f40-86de-b9c54d323591" containerName="cleanup"
Jul 23 23:38:34.238139 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:34.238102    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc5db127-44bc-4f40-86de-b9c54d323591" containerName="cleanup"
Jul 23 23:38:34.241182 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:34.241166    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4"
Jul 23 23:38:34.244014 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:34.243990    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-8f14c64d-openshift-default-dockercfg-xdc4w\""
Jul 23 23:38:34.261494 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:34.261466    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4"]
Jul 23 23:38:34.340714 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:34.340625    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xr56s\" (UniqueName: \"kubernetes.io/projected/96fceab3-3092-4d94-bcd3-df960292f3a8-kube-api-access-xr56s\") pod \"e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4\" (UID: \"96fceab3-3092-4d94-bcd3-df960292f3a8\") " pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4"
Jul 23 23:38:34.340919 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:34.340751    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96fceab3-3092-4d94-bcd3-df960292f3a8-workload-certs\") pod \"e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4\" (UID: \"96fceab3-3092-4d94-bcd3-df960292f3a8\") " pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4"
Jul 23 23:38:34.340919 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:34.340881    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96fceab3-3092-4d94-bcd3-df960292f3a8-workload-socket\") pod \"e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4\" (UID: \"96fceab3-3092-4d94-bcd3-df960292f3a8\") " pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4"
Jul 23 23:38:34.340919 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:34.340913    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96fceab3-3092-4d94-bcd3-df960292f3a8-credential-socket\") pod \"e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4\" (UID: \"96fceab3-3092-4d94-bcd3-df960292f3a8\") " pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4"
Jul 23 23:38:34.341091 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:34.340948    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96fceab3-3092-4d94-bcd3-df960292f3a8-istio-data\") pod \"e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4\" (UID: \"96fceab3-3092-4d94-bcd3-df960292f3a8\") " pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4"
Jul 23 23:38:34.341091 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:34.340975    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96fceab3-3092-4d94-bcd3-df960292f3a8-istiod-ca-cert\") pod \"e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4\" (UID: \"96fceab3-3092-4d94-bcd3-df960292f3a8\") " pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4"
Jul 23 23:38:34.341091 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:34.341039    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96fceab3-3092-4d94-bcd3-df960292f3a8-istio-podinfo\") pod \"e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4\" (UID: \"96fceab3-3092-4d94-bcd3-df960292f3a8\") " pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4"
Jul 23 23:38:34.341091 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:34.341067    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96fceab3-3092-4d94-bcd3-df960292f3a8-istio-token\") pod \"e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4\" (UID: \"96fceab3-3092-4d94-bcd3-df960292f3a8\") " pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4"
Jul 23 23:38:34.341272 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:34.341179    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96fceab3-3092-4d94-bcd3-df960292f3a8-istio-envoy\") pod \"e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4\" (UID: \"96fceab3-3092-4d94-bcd3-df960292f3a8\") " pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4"
Jul 23 23:38:34.442604 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:34.442559    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96fceab3-3092-4d94-bcd3-df960292f3a8-credential-socket\") pod \"e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4\" (UID: \"96fceab3-3092-4d94-bcd3-df960292f3a8\") " pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4"
Jul 23 23:38:34.442845 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:34.442616    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96fceab3-3092-4d94-bcd3-df960292f3a8-istio-data\") pod \"e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4\" (UID: \"96fceab3-3092-4d94-bcd3-df960292f3a8\") " pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4"
Jul 23 23:38:34.442845 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:34.442643    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96fceab3-3092-4d94-bcd3-df960292f3a8-istiod-ca-cert\") pod \"e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4\" (UID: \"96fceab3-3092-4d94-bcd3-df960292f3a8\") " pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4"
Jul 23 23:38:34.442845 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:34.442719    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96fceab3-3092-4d94-bcd3-df960292f3a8-istio-podinfo\") pod \"e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4\" (UID: \"96fceab3-3092-4d94-bcd3-df960292f3a8\") " pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4"
Jul 23 23:38:34.442845 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:34.442747    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96fceab3-3092-4d94-bcd3-df960292f3a8-istio-token\") pod \"e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4\" (UID: \"96fceab3-3092-4d94-bcd3-df960292f3a8\") " pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4"
Jul 23 23:38:34.442845 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:34.442833    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96fceab3-3092-4d94-bcd3-df960292f3a8-istio-envoy\") pod \"e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4\" (UID: \"96fceab3-3092-4d94-bcd3-df960292f3a8\") " pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4"
Jul 23 23:38:34.443136 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:34.442866    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xr56s\" (UniqueName: \"kubernetes.io/projected/96fceab3-3092-4d94-bcd3-df960292f3a8-kube-api-access-xr56s\") pod \"e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4\" (UID: \"96fceab3-3092-4d94-bcd3-df960292f3a8\") " pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4"
Jul 23 23:38:34.443136 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:34.442893    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96fceab3-3092-4d94-bcd3-df960292f3a8-workload-certs\") pod \"e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4\" (UID: \"96fceab3-3092-4d94-bcd3-df960292f3a8\") " pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4"
Jul 23 23:38:34.443136 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:34.442988    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96fceab3-3092-4d94-bcd3-df960292f3a8-workload-socket\") pod \"e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4\" (UID: \"96fceab3-3092-4d94-bcd3-df960292f3a8\") " pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4"
Jul 23 23:38:34.443136 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:34.443060    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96fceab3-3092-4d94-bcd3-df960292f3a8-credential-socket\") pod \"e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4\" (UID: \"96fceab3-3092-4d94-bcd3-df960292f3a8\") " pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4"
Jul 23 23:38:34.443333 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:34.443146    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96fceab3-3092-4d94-bcd3-df960292f3a8-istio-data\") pod \"e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4\" (UID: \"96fceab3-3092-4d94-bcd3-df960292f3a8\") " pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4"
Jul 23 23:38:34.443394 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:34.443364    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96fceab3-3092-4d94-bcd3-df960292f3a8-workload-socket\") pod \"e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4\" (UID: \"96fceab3-3092-4d94-bcd3-df960292f3a8\") " pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4"
Jul 23 23:38:34.443449 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:34.443423    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96fceab3-3092-4d94-bcd3-df960292f3a8-workload-certs\") pod \"e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4\" (UID: \"96fceab3-3092-4d94-bcd3-df960292f3a8\") " pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4"
Jul 23 23:38:34.443604 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:34.443555    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96fceab3-3092-4d94-bcd3-df960292f3a8-istiod-ca-cert\") pod \"e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4\" (UID: \"96fceab3-3092-4d94-bcd3-df960292f3a8\") " pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4"
Jul 23 23:38:34.445680 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:34.445623    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96fceab3-3092-4d94-bcd3-df960292f3a8-istio-envoy\") pod \"e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4\" (UID: \"96fceab3-3092-4d94-bcd3-df960292f3a8\") " pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4"
Jul 23 23:38:34.445920 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:34.445905    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96fceab3-3092-4d94-bcd3-df960292f3a8-istio-podinfo\") pod \"e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4\" (UID: \"96fceab3-3092-4d94-bcd3-df960292f3a8\") " pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4"
Jul 23 23:38:34.451614 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:34.451590    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96fceab3-3092-4d94-bcd3-df960292f3a8-istio-token\") pod \"e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4\" (UID: \"96fceab3-3092-4d94-bcd3-df960292f3a8\") " pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4"
Jul 23 23:38:34.452140 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:34.452119    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xr56s\" (UniqueName: \"kubernetes.io/projected/96fceab3-3092-4d94-bcd3-df960292f3a8-kube-api-access-xr56s\") pod \"e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4\" (UID: \"96fceab3-3092-4d94-bcd3-df960292f3a8\") " pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4"
Jul 23 23:38:34.551563 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:34.551465    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4"
Jul 23 23:38:34.710260 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:34.710228    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4"]
Jul 23 23:38:34.711955 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:38:34.711930    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96fceab3_3092_4d94_bcd3_df960292f3a8.slice/crio-216143853ce38bbbe3b700accdff17bacf0be283466354354ad612106e0b5196 WatchSource:0}: Error finding container 216143853ce38bbbe3b700accdff17bacf0be283466354354ad612106e0b5196: Status 404 returned error can't find the container with id 216143853ce38bbbe3b700accdff17bacf0be283466354354ad612106e0b5196
Jul 23 23:38:34.714602 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:34.714572    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:38:34.714719 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:34.714646    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:38:34.714780 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:34.714744    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:38:35.123019 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:35.122981    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4" event={"ID":"96fceab3-3092-4d94-bcd3-df960292f3a8","Type":"ContainerStarted","Data":"9fc541d7a2078b5d85533fddc02dddf0bfb95194821bf70bcc2c704027168926"}
Jul 23 23:38:35.123218 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:35.123035    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4" event={"ID":"96fceab3-3092-4d94-bcd3-df960292f3a8","Type":"ContainerStarted","Data":"216143853ce38bbbe3b700accdff17bacf0be283466354354ad612106e0b5196"}
Jul 23 23:38:35.143547 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:35.143487    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4" podStartSLOduration=1.1434652459999999 podStartE2EDuration="1.143465246s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:38:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:38:35.142747967 +0000 UTC m=+1504.512273628" watchObservedRunningTime="2026-07-23 23:38:35.143465246 +0000 UTC m=+1504.512990905"
Jul 23 23:38:35.551756 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:35.551710    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4"
Jul 23 23:38:35.557455 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:35.557426    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4"
Jul 23 23:38:36.127860 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:36.127829    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4"
Jul 23 23:38:36.129011 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:36.128987    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4"
Jul 23 23:38:36.553023 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:36.552983    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8f14c64d-5d44c76d59-d9n8j"]
Jul 23 23:38:36.556814 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:36.556795    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f14c64d-5d44c76d59-d9n8j"
Jul 23 23:38:36.559586 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:36.559564    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-8f14c64d-serving-cert\""
Jul 23 23:38:36.568792 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:36.568765    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8f14c64d-5d44c76d59-d9n8j"]
Jul 23 23:38:36.661452 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:36.661289    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/014f05db-82d1-4385-afb0-247566ff1064-maas-api-tls\") pod \"maas-api-e2e-ait-8f14c64d-5d44c76d59-d9n8j\" (UID: \"014f05db-82d1-4385-afb0-247566ff1064\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f14c64d-5d44c76d59-d9n8j"
Jul 23 23:38:36.661690 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:36.661485    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/014f05db-82d1-4385-afb0-247566ff1064-openshift-service-ca\") pod \"maas-api-e2e-ait-8f14c64d-5d44c76d59-d9n8j\" (UID: \"014f05db-82d1-4385-afb0-247566ff1064\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f14c64d-5d44c76d59-d9n8j"
Jul 23 23:38:36.661809 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:36.661763    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6flc\" (UniqueName: \"kubernetes.io/projected/014f05db-82d1-4385-afb0-247566ff1064-kube-api-access-b6flc\") pod \"maas-api-e2e-ait-8f14c64d-5d44c76d59-d9n8j\" (UID: \"014f05db-82d1-4385-afb0-247566ff1064\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f14c64d-5d44c76d59-d9n8j"
Jul 23 23:38:36.763156 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:36.763123    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6flc\" (UniqueName: \"kubernetes.io/projected/014f05db-82d1-4385-afb0-247566ff1064-kube-api-access-b6flc\") pod \"maas-api-e2e-ait-8f14c64d-5d44c76d59-d9n8j\" (UID: \"014f05db-82d1-4385-afb0-247566ff1064\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f14c64d-5d44c76d59-d9n8j"
Jul 23 23:38:36.763156 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:36.763161    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/014f05db-82d1-4385-afb0-247566ff1064-maas-api-tls\") pod \"maas-api-e2e-ait-8f14c64d-5d44c76d59-d9n8j\" (UID: \"014f05db-82d1-4385-afb0-247566ff1064\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f14c64d-5d44c76d59-d9n8j"
Jul 23 23:38:36.763375 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:36.763182    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/014f05db-82d1-4385-afb0-247566ff1064-openshift-service-ca\") pod \"maas-api-e2e-ait-8f14c64d-5d44c76d59-d9n8j\" (UID: \"014f05db-82d1-4385-afb0-247566ff1064\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f14c64d-5d44c76d59-d9n8j"
Jul 23 23:38:36.763375 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:38:36.763281    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-8f14c64d-serving-cert: secret "maas-api-e2e-ait-8f14c64d-serving-cert" not found
Jul 23 23:38:36.763375 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:38:36.763355    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/014f05db-82d1-4385-afb0-247566ff1064-maas-api-tls podName:014f05db-82d1-4385-afb0-247566ff1064 nodeName:}" failed. No retries permitted until 2026-07-23 23:38:37.26333543 +0000 UTC m=+1506.632861070 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/014f05db-82d1-4385-afb0-247566ff1064-maas-api-tls") pod "maas-api-e2e-ait-8f14c64d-5d44c76d59-d9n8j" (UID: "014f05db-82d1-4385-afb0-247566ff1064") : secret "maas-api-e2e-ait-8f14c64d-serving-cert" not found
Jul 23 23:38:36.763936 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:36.763911    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/014f05db-82d1-4385-afb0-247566ff1064-openshift-service-ca\") pod \"maas-api-e2e-ait-8f14c64d-5d44c76d59-d9n8j\" (UID: \"014f05db-82d1-4385-afb0-247566ff1064\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f14c64d-5d44c76d59-d9n8j"
Jul 23 23:38:36.772731 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:36.772707    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6flc\" (UniqueName: \"kubernetes.io/projected/014f05db-82d1-4385-afb0-247566ff1064-kube-api-access-b6flc\") pod \"maas-api-e2e-ait-8f14c64d-5d44c76d59-d9n8j\" (UID: \"014f05db-82d1-4385-afb0-247566ff1064\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f14c64d-5d44c76d59-d9n8j"
Jul 23 23:38:37.267776 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:37.267719    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/014f05db-82d1-4385-afb0-247566ff1064-maas-api-tls\") pod \"maas-api-e2e-ait-8f14c64d-5d44c76d59-d9n8j\" (UID: \"014f05db-82d1-4385-afb0-247566ff1064\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f14c64d-5d44c76d59-d9n8j"
Jul 23 23:38:37.270366 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:37.270338    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/014f05db-82d1-4385-afb0-247566ff1064-maas-api-tls\") pod \"maas-api-e2e-ait-8f14c64d-5d44c76d59-d9n8j\" (UID: \"014f05db-82d1-4385-afb0-247566ff1064\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f14c64d-5d44c76d59-d9n8j"
Jul 23 23:38:37.470281 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:37.470244    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f14c64d-5d44c76d59-d9n8j"
Jul 23 23:38:37.611569 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:37.611539    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8f14c64d-5d44c76d59-d9n8j"]
Jul 23 23:38:37.613767 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:38:37.613733    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod014f05db_82d1_4385_afb0_247566ff1064.slice/crio-1915c36d0785d47fb4855174acde60d6f8695b37f3ec69095606efc25476be9f WatchSource:0}: Error finding container 1915c36d0785d47fb4855174acde60d6f8695b37f3ec69095606efc25476be9f: Status 404 returned error can't find the container with id 1915c36d0785d47fb4855174acde60d6f8695b37f3ec69095606efc25476be9f
Jul 23 23:38:37.615309 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:37.615288    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 23:38:38.139317 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:38.139276    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f14c64d-5d44c76d59-d9n8j" event={"ID":"014f05db-82d1-4385-afb0-247566ff1064","Type":"ContainerStarted","Data":"8defeaf7914dace2a16245b034a7564c8ed55ec913f28d895bac9f5663666513"}
Jul 23 23:38:38.139517 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:38.139339    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f14c64d-5d44c76d59-d9n8j" event={"ID":"014f05db-82d1-4385-afb0-247566ff1064","Type":"ContainerStarted","Data":"1915c36d0785d47fb4855174acde60d6f8695b37f3ec69095606efc25476be9f"}
Jul 23 23:38:38.139517 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:38.139439    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f14c64d-5d44c76d59-d9n8j"
Jul 23 23:38:38.158930 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:38.158873    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f14c64d-5d44c76d59-d9n8j" podStartSLOduration=1.76918438 podStartE2EDuration="2.158855633s" totalImagesPullingTime="389.671253ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:38:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:38:38.15712946 +0000 UTC m=+1507.526655133" watchObservedRunningTime="2026-07-23 23:38:38.158855633 +0000 UTC m=+1507.528381295"
Jul 23 23:38:44.151126 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:44.151092    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f14c64d-5d44c76d59-d9n8j"
Jul 23 23:38:51.440921 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:51.440814    2578 patch_prober.go:28] interesting pod/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": read tcp 10.134.0.2:35406->10.134.0.31:15021: read: connection reset by peer" start-of-body=
Jul 23 23:38:51.440921 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:51.440892    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4" podUID="96fceab3-3092-4d94-bcd3-df960292f3a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": read tcp 10.134.0.2:35406->10.134.0.31:15021: read: connection reset by peer"
Jul 23 23:38:52.222108 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:52.222072    2578 generic.go:356] "Generic (PLEG): container finished" podID="96fceab3-3092-4d94-bcd3-df960292f3a8" containerID="9fc541d7a2078b5d85533fddc02dddf0bfb95194821bf70bcc2c704027168926" exitCode=0
Jul 23 23:38:52.222293 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:52.222144    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4" event={"ID":"96fceab3-3092-4d94-bcd3-df960292f3a8","Type":"ContainerDied","Data":"9fc541d7a2078b5d85533fddc02dddf0bfb95194821bf70bcc2c704027168926"}
Jul 23 23:38:52.222552 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:52.222537    2578 scope.go:117] "RemoveContainer" containerID="9fc541d7a2078b5d85533fddc02dddf0bfb95194821bf70bcc2c704027168926"
Jul 23 23:38:52.223530 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:52.223506    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:38:52.223582 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:52.223573    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:38:52.223620 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:52.223614    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:38:53.229055 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:53.229021    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4" event={"ID":"96fceab3-3092-4d94-bcd3-df960292f3a8","Type":"ContainerStarted","Data":"4b267ea71d2dc5d16eec3ef9a67acd1b4612d3673930c2ecfbb33252bb5d0f7c"}
Jul 23 23:38:53.552468 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:53.552358    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4"
Jul 23 23:38:53.552636 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:53.552610    2578 patch_prober.go:28] interesting pod/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Jul 23 23:38:53.552718 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:53.552696    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4" podUID="96fceab3-3092-4d94-bcd3-df960292f3a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Jul 23 23:38:54.552246 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:54.552202    2578 patch_prober.go:28] interesting pod/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Jul 23 23:38:54.552750 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:54.552267    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4" podUID="96fceab3-3092-4d94-bcd3-df960292f3a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Jul 23 23:38:55.552374 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:55.552334    2578 patch_prober.go:28] interesting pod/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Jul 23 23:38:55.552780 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:55.552392    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4" podUID="96fceab3-3092-4d94-bcd3-df960292f3a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Jul 23 23:38:56.552131 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:56.552091    2578 patch_prober.go:28] interesting pod/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Jul 23 23:38:56.552321 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:56.552153    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4" podUID="96fceab3-3092-4d94-bcd3-df960292f3a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Jul 23 23:38:57.555879 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:57.555845    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4"
Jul 23 23:38:57.556273 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:57.556212    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4"
Jul 23 23:38:57.556851 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:38:57.556836    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4"
Jul 23 23:39:12.609289 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:12.609254    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8f14c64d-5d44c76d59-d9n8j"]
Jul 23 23:39:12.610078 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:12.610026    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f14c64d-5d44c76d59-d9n8j" podUID="014f05db-82d1-4385-afb0-247566ff1064" containerName="maas-api" containerID="cri-o://8defeaf7914dace2a16245b034a7564c8ed55ec913f28d895bac9f5663666513" gracePeriod=30
Jul 23 23:39:12.864587 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:12.864525    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f14c64d-5d44c76d59-d9n8j"
Jul 23 23:39:12.930446 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:12.930404    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b6flc\" (UniqueName: \"kubernetes.io/projected/014f05db-82d1-4385-afb0-247566ff1064-kube-api-access-b6flc\") pod \"014f05db-82d1-4385-afb0-247566ff1064\" (UID: \"014f05db-82d1-4385-afb0-247566ff1064\") "
Jul 23 23:39:12.930640 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:12.930493    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/014f05db-82d1-4385-afb0-247566ff1064-maas-api-tls\") pod \"014f05db-82d1-4385-afb0-247566ff1064\" (UID: \"014f05db-82d1-4385-afb0-247566ff1064\") "
Jul 23 23:39:12.930640 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:12.930533    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/014f05db-82d1-4385-afb0-247566ff1064-openshift-service-ca\") pod \"014f05db-82d1-4385-afb0-247566ff1064\" (UID: \"014f05db-82d1-4385-afb0-247566ff1064\") "
Jul 23 23:39:12.930960 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:12.930929    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/014f05db-82d1-4385-afb0-247566ff1064-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "014f05db-82d1-4385-afb0-247566ff1064" (UID: "014f05db-82d1-4385-afb0-247566ff1064"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:39:12.932913 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:12.932888    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/014f05db-82d1-4385-afb0-247566ff1064-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "014f05db-82d1-4385-afb0-247566ff1064" (UID: "014f05db-82d1-4385-afb0-247566ff1064"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 23:39:12.933000 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:12.932967    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/014f05db-82d1-4385-afb0-247566ff1064-kube-api-access-b6flc" (OuterVolumeSpecName: "kube-api-access-b6flc") pod "014f05db-82d1-4385-afb0-247566ff1064" (UID: "014f05db-82d1-4385-afb0-247566ff1064"). InnerVolumeSpecName "kube-api-access-b6flc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:39:13.031202 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:13.031159    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/014f05db-82d1-4385-afb0-247566ff1064-maas-api-tls\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:39:13.031202 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:13.031196    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/014f05db-82d1-4385-afb0-247566ff1064-openshift-service-ca\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:39:13.031202 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:13.031205    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b6flc\" (UniqueName: \"kubernetes.io/projected/014f05db-82d1-4385-afb0-247566ff1064-kube-api-access-b6flc\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:39:13.322402 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:13.322367    2578 generic.go:356] "Generic (PLEG): container finished" podID="014f05db-82d1-4385-afb0-247566ff1064" containerID="8defeaf7914dace2a16245b034a7564c8ed55ec913f28d895bac9f5663666513" exitCode=0
Jul 23 23:39:13.322567 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:13.322414    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f14c64d-5d44c76d59-d9n8j"
Jul 23 23:39:13.322567 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:13.322457    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f14c64d-5d44c76d59-d9n8j" event={"ID":"014f05db-82d1-4385-afb0-247566ff1064","Type":"ContainerDied","Data":"8defeaf7914dace2a16245b034a7564c8ed55ec913f28d895bac9f5663666513"}
Jul 23 23:39:13.322567 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:13.322512    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f14c64d-5d44c76d59-d9n8j" event={"ID":"014f05db-82d1-4385-afb0-247566ff1064","Type":"ContainerDied","Data":"1915c36d0785d47fb4855174acde60d6f8695b37f3ec69095606efc25476be9f"}
Jul 23 23:39:13.322567 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:13.322542    2578 scope.go:117] "RemoveContainer" containerID="8defeaf7914dace2a16245b034a7564c8ed55ec913f28d895bac9f5663666513"
Jul 23 23:39:13.331046 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:13.331026    2578 scope.go:117] "RemoveContainer" containerID="8defeaf7914dace2a16245b034a7564c8ed55ec913f28d895bac9f5663666513"
Jul 23 23:39:13.331290 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:39:13.331273    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8defeaf7914dace2a16245b034a7564c8ed55ec913f28d895bac9f5663666513\": container with ID starting with 8defeaf7914dace2a16245b034a7564c8ed55ec913f28d895bac9f5663666513 not found: ID does not exist" containerID="8defeaf7914dace2a16245b034a7564c8ed55ec913f28d895bac9f5663666513"
Jul 23 23:39:13.331352 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:13.331298    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8defeaf7914dace2a16245b034a7564c8ed55ec913f28d895bac9f5663666513"} err="failed to get container status \"8defeaf7914dace2a16245b034a7564c8ed55ec913f28d895bac9f5663666513\": rpc error: code = NotFound desc = could not find container \"8defeaf7914dace2a16245b034a7564c8ed55ec913f28d895bac9f5663666513\": container with ID starting with 8defeaf7914dace2a16245b034a7564c8ed55ec913f28d895bac9f5663666513 not found: ID does not exist"
Jul 23 23:39:13.345507 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:13.345436    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8f14c64d-5d44c76d59-d9n8j"]
Jul 23 23:39:13.346385 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:13.346362    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8f14c64d-5d44c76d59-d9n8j"]
Jul 23 23:39:15.236381 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:15.236351    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="014f05db-82d1-4385-afb0-247566ff1064" path="/var/lib/kubelet/pods/014f05db-82d1-4385-afb0-247566ff1064/volumes"
Jul 23 23:39:17.314296 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:17.314261    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4"]
Jul 23 23:39:17.314811 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:17.314481    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4" podUID="96fceab3-3092-4d94-bcd3-df960292f3a8" containerName="istio-proxy" containerID="cri-o://4b267ea71d2dc5d16eec3ef9a67acd1b4612d3673930c2ecfbb33252bb5d0f7c" gracePeriod=30
Jul 23 23:39:18.404619 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:18.404581    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"]
Jul 23 23:39:18.405195 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:18.405174    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="014f05db-82d1-4385-afb0-247566ff1064" containerName="maas-api"
Jul 23 23:39:18.405247 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:18.405200    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="014f05db-82d1-4385-afb0-247566ff1064" containerName="maas-api"
Jul 23 23:39:18.405335 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:18.405322    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="014f05db-82d1-4385-afb0-247566ff1064" containerName="maas-api"
Jul 23 23:39:18.410026 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:18.410008    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"
Jul 23 23:39:18.412732 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:18.412711    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-1372ee67-gw-openshift-default-dockercfg-5p944\""
Jul 23 23:39:18.426411 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:18.426375    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"]
Jul 23 23:39:18.484780 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:18.484737    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxtb7\" (UniqueName: \"kubernetes.io/projected/6738398f-fc24-43b6-b213-21b1cf59eb01-kube-api-access-rxtb7\") pod \"e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh\" (UID: \"6738398f-fc24-43b6-b213-21b1cf59eb01\") " pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"
Jul 23 23:39:18.484963 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:18.484808    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6738398f-fc24-43b6-b213-21b1cf59eb01-istio-token\") pod \"e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh\" (UID: \"6738398f-fc24-43b6-b213-21b1cf59eb01\") " pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"
Jul 23 23:39:18.484963 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:18.484900    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6738398f-fc24-43b6-b213-21b1cf59eb01-credential-socket\") pod \"e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh\" (UID: \"6738398f-fc24-43b6-b213-21b1cf59eb01\") " pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"
Jul 23 23:39:18.484963 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:18.484946    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6738398f-fc24-43b6-b213-21b1cf59eb01-workload-certs\") pod \"e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh\" (UID: \"6738398f-fc24-43b6-b213-21b1cf59eb01\") " pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"
Jul 23 23:39:18.485162 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:18.484965    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6738398f-fc24-43b6-b213-21b1cf59eb01-workload-socket\") pod \"e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh\" (UID: \"6738398f-fc24-43b6-b213-21b1cf59eb01\") " pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"
Jul 23 23:39:18.485162 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:18.484990    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6738398f-fc24-43b6-b213-21b1cf59eb01-istiod-ca-cert\") pod \"e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh\" (UID: \"6738398f-fc24-43b6-b213-21b1cf59eb01\") " pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"
Jul 23 23:39:18.485162 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:18.485041    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6738398f-fc24-43b6-b213-21b1cf59eb01-istio-envoy\") pod \"e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh\" (UID: \"6738398f-fc24-43b6-b213-21b1cf59eb01\") " pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"
Jul 23 23:39:18.485162 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:18.485059    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6738398f-fc24-43b6-b213-21b1cf59eb01-istio-data\") pod \"e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh\" (UID: \"6738398f-fc24-43b6-b213-21b1cf59eb01\") " pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"
Jul 23 23:39:18.485336 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:18.485154    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6738398f-fc24-43b6-b213-21b1cf59eb01-istio-podinfo\") pod \"e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh\" (UID: \"6738398f-fc24-43b6-b213-21b1cf59eb01\") " pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"
Jul 23 23:39:18.586633 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:18.586588    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxtb7\" (UniqueName: \"kubernetes.io/projected/6738398f-fc24-43b6-b213-21b1cf59eb01-kube-api-access-rxtb7\") pod \"e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh\" (UID: \"6738398f-fc24-43b6-b213-21b1cf59eb01\") " pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"
Jul 23 23:39:18.586845 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:18.586683    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6738398f-fc24-43b6-b213-21b1cf59eb01-istio-token\") pod \"e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh\" (UID: \"6738398f-fc24-43b6-b213-21b1cf59eb01\") " pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"
Jul 23 23:39:18.586845 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:18.586739    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6738398f-fc24-43b6-b213-21b1cf59eb01-credential-socket\") pod \"e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh\" (UID: \"6738398f-fc24-43b6-b213-21b1cf59eb01\") " pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"
Jul 23 23:39:18.586986 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:18.586956    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6738398f-fc24-43b6-b213-21b1cf59eb01-workload-certs\") pod \"e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh\" (UID: \"6738398f-fc24-43b6-b213-21b1cf59eb01\") " pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"
Jul 23 23:39:18.587049 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:18.587006    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6738398f-fc24-43b6-b213-21b1cf59eb01-workload-socket\") pod \"e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh\" (UID: \"6738398f-fc24-43b6-b213-21b1cf59eb01\") " pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"
Jul 23 23:39:18.587104 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:18.587048    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6738398f-fc24-43b6-b213-21b1cf59eb01-istiod-ca-cert\") pod \"e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh\" (UID: \"6738398f-fc24-43b6-b213-21b1cf59eb01\") " pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"
Jul 23 23:39:18.587104 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:18.587084    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6738398f-fc24-43b6-b213-21b1cf59eb01-istio-envoy\") pod \"e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh\" (UID: \"6738398f-fc24-43b6-b213-21b1cf59eb01\") " pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"
Jul 23 23:39:18.587212 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:18.587112    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6738398f-fc24-43b6-b213-21b1cf59eb01-istio-data\") pod \"e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh\" (UID: \"6738398f-fc24-43b6-b213-21b1cf59eb01\") " pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"
Jul 23 23:39:18.587212 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:18.587152    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6738398f-fc24-43b6-b213-21b1cf59eb01-istio-podinfo\") pod \"e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh\" (UID: \"6738398f-fc24-43b6-b213-21b1cf59eb01\") " pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"
Jul 23 23:39:18.587443 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:18.587331    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6738398f-fc24-43b6-b213-21b1cf59eb01-credential-socket\") pod \"e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh\" (UID: \"6738398f-fc24-43b6-b213-21b1cf59eb01\") " pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"
Jul 23 23:39:18.587539 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:18.587484    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6738398f-fc24-43b6-b213-21b1cf59eb01-workload-socket\") pod \"e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh\" (UID: \"6738398f-fc24-43b6-b213-21b1cf59eb01\") " pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"
Jul 23 23:39:18.587798 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:18.587774    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6738398f-fc24-43b6-b213-21b1cf59eb01-workload-certs\") pod \"e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh\" (UID: \"6738398f-fc24-43b6-b213-21b1cf59eb01\") " pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"
Jul 23 23:39:18.587938 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:18.587862    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6738398f-fc24-43b6-b213-21b1cf59eb01-istio-data\") pod \"e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh\" (UID: \"6738398f-fc24-43b6-b213-21b1cf59eb01\") " pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"
Jul 23 23:39:18.587938 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:18.587881    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6738398f-fc24-43b6-b213-21b1cf59eb01-istiod-ca-cert\") pod \"e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh\" (UID: \"6738398f-fc24-43b6-b213-21b1cf59eb01\") " pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"
Jul 23 23:39:18.591155 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:18.590687    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6738398f-fc24-43b6-b213-21b1cf59eb01-istio-envoy\") pod \"e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh\" (UID: \"6738398f-fc24-43b6-b213-21b1cf59eb01\") " pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"
Jul 23 23:39:18.591155 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:18.590946    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6738398f-fc24-43b6-b213-21b1cf59eb01-istio-podinfo\") pod \"e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh\" (UID: \"6738398f-fc24-43b6-b213-21b1cf59eb01\") " pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"
Jul 23 23:39:18.596911 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:18.596836    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6738398f-fc24-43b6-b213-21b1cf59eb01-istio-token\") pod \"e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh\" (UID: \"6738398f-fc24-43b6-b213-21b1cf59eb01\") " pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"
Jul 23 23:39:18.597072 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:18.596994    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxtb7\" (UniqueName: \"kubernetes.io/projected/6738398f-fc24-43b6-b213-21b1cf59eb01-kube-api-access-rxtb7\") pod \"e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh\" (UID: \"6738398f-fc24-43b6-b213-21b1cf59eb01\") " pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"
Jul 23 23:39:18.721608 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:18.721575    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"
Jul 23 23:39:18.864081 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:18.863950    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"]
Jul 23 23:39:18.866237 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:39:18.866213    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6738398f_fc24_43b6_b213_21b1cf59eb01.slice/crio-13de3e427c58d8e397b45cd524bcf4cc9edf589e0ec045eb0cd75ccef115b6ee WatchSource:0}: Error finding container 13de3e427c58d8e397b45cd524bcf4cc9edf589e0ec045eb0cd75ccef115b6ee: Status 404 returned error can't find the container with id 13de3e427c58d8e397b45cd524bcf4cc9edf589e0ec045eb0cd75ccef115b6ee
Jul 23 23:39:18.868384 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:18.868352    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:39:18.868457 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:18.868419    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:39:18.868457 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:18.868452    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:39:19.355973 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:19.355936    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh" event={"ID":"6738398f-fc24-43b6-b213-21b1cf59eb01","Type":"ContainerStarted","Data":"beb6690c86789c43464f5b3483bca1689d2d272e6e01dc2731a3821daef6ec8e"}
Jul 23 23:39:19.356151 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:19.355989    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh" event={"ID":"6738398f-fc24-43b6-b213-21b1cf59eb01","Type":"ContainerStarted","Data":"13de3e427c58d8e397b45cd524bcf4cc9edf589e0ec045eb0cd75ccef115b6ee"}
Jul 23 23:39:19.378577 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:19.378523    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh" podStartSLOduration=1.378504208 podStartE2EDuration="1.378504208s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:39:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:39:19.378337761 +0000 UTC m=+1548.747863422" watchObservedRunningTime="2026-07-23 23:39:19.378504208 +0000 UTC m=+1548.748029869"
Jul 23 23:39:19.722004 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:19.721963    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"
Jul 23 23:39:19.726877 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:19.726852    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"
Jul 23 23:39:20.360419 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:20.360384    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"
Jul 23 23:39:20.361511 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:20.361490    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"
Jul 23 23:39:20.995826 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:20.995785    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-1372ee67-66d46b9668-n9mhd"]
Jul 23 23:39:21.000045 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:21.000021    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1372ee67-66d46b9668-n9mhd"
Jul 23 23:39:21.002818 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:21.002792    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-1372ee67-serving-cert\""
Jul 23 23:39:21.009139 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:21.009116    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-1372ee67-66d46b9668-n9mhd"]
Jul 23 23:39:21.114087 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:21.114039    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bebec368-bca0-40ac-9dda-b31c3286b2b7-openshift-service-ca\") pod \"maas-api-e2e-migrate-1372ee67-66d46b9668-n9mhd\" (UID: \"bebec368-bca0-40ac-9dda-b31c3286b2b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1372ee67-66d46b9668-n9mhd"
Jul 23 23:39:21.114252 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:21.114102    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tbtp\" (UniqueName: \"kubernetes.io/projected/bebec368-bca0-40ac-9dda-b31c3286b2b7-kube-api-access-9tbtp\") pod \"maas-api-e2e-migrate-1372ee67-66d46b9668-n9mhd\" (UID: \"bebec368-bca0-40ac-9dda-b31c3286b2b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1372ee67-66d46b9668-n9mhd"
Jul 23 23:39:21.114252 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:21.114218    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bebec368-bca0-40ac-9dda-b31c3286b2b7-maas-api-tls\") pod \"maas-api-e2e-migrate-1372ee67-66d46b9668-n9mhd\" (UID: \"bebec368-bca0-40ac-9dda-b31c3286b2b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1372ee67-66d46b9668-n9mhd"
Jul 23 23:39:21.214973 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:21.214938    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bebec368-bca0-40ac-9dda-b31c3286b2b7-maas-api-tls\") pod \"maas-api-e2e-migrate-1372ee67-66d46b9668-n9mhd\" (UID: \"bebec368-bca0-40ac-9dda-b31c3286b2b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1372ee67-66d46b9668-n9mhd"
Jul 23 23:39:21.215144 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:21.215006    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bebec368-bca0-40ac-9dda-b31c3286b2b7-openshift-service-ca\") pod \"maas-api-e2e-migrate-1372ee67-66d46b9668-n9mhd\" (UID: \"bebec368-bca0-40ac-9dda-b31c3286b2b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1372ee67-66d46b9668-n9mhd"
Jul 23 23:39:21.215144 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:21.215043    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tbtp\" (UniqueName: \"kubernetes.io/projected/bebec368-bca0-40ac-9dda-b31c3286b2b7-kube-api-access-9tbtp\") pod \"maas-api-e2e-migrate-1372ee67-66d46b9668-n9mhd\" (UID: \"bebec368-bca0-40ac-9dda-b31c3286b2b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1372ee67-66d46b9668-n9mhd"
Jul 23 23:39:21.215144 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:39:21.215089    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-1372ee67-serving-cert: secret "maas-api-e2e-migrate-1372ee67-serving-cert" not found
Jul 23 23:39:21.215240 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:39:21.215152    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bebec368-bca0-40ac-9dda-b31c3286b2b7-maas-api-tls podName:bebec368-bca0-40ac-9dda-b31c3286b2b7 nodeName:}" failed. No retries permitted until 2026-07-23 23:39:21.715136232 +0000 UTC m=+1551.084661868 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bebec368-bca0-40ac-9dda-b31c3286b2b7-maas-api-tls") pod "maas-api-e2e-migrate-1372ee67-66d46b9668-n9mhd" (UID: "bebec368-bca0-40ac-9dda-b31c3286b2b7") : secret "maas-api-e2e-migrate-1372ee67-serving-cert" not found
Jul 23 23:39:21.215790 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:21.215772    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bebec368-bca0-40ac-9dda-b31c3286b2b7-openshift-service-ca\") pod \"maas-api-e2e-migrate-1372ee67-66d46b9668-n9mhd\" (UID: \"bebec368-bca0-40ac-9dda-b31c3286b2b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1372ee67-66d46b9668-n9mhd"
Jul 23 23:39:21.225022 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:21.224992    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tbtp\" (UniqueName: \"kubernetes.io/projected/bebec368-bca0-40ac-9dda-b31c3286b2b7-kube-api-access-9tbtp\") pod \"maas-api-e2e-migrate-1372ee67-66d46b9668-n9mhd\" (UID: \"bebec368-bca0-40ac-9dda-b31c3286b2b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1372ee67-66d46b9668-n9mhd"
Jul 23 23:39:21.720693 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:21.720640    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bebec368-bca0-40ac-9dda-b31c3286b2b7-maas-api-tls\") pod \"maas-api-e2e-migrate-1372ee67-66d46b9668-n9mhd\" (UID: \"bebec368-bca0-40ac-9dda-b31c3286b2b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1372ee67-66d46b9668-n9mhd"
Jul 23 23:39:21.723280 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:21.723262    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bebec368-bca0-40ac-9dda-b31c3286b2b7-maas-api-tls\") pod \"maas-api-e2e-migrate-1372ee67-66d46b9668-n9mhd\" (UID: \"bebec368-bca0-40ac-9dda-b31c3286b2b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1372ee67-66d46b9668-n9mhd"
Jul 23 23:39:21.911989 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:21.911947    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1372ee67-66d46b9668-n9mhd"
Jul 23 23:39:22.257495 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:22.257467    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-1372ee67-66d46b9668-n9mhd"]
Jul 23 23:39:22.260026 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:39:22.259999    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbebec368_bca0_40ac_9dda_b31c3286b2b7.slice/crio-310cf26a01694134856c908235959e67e21a715877a7b7f938a6796ec2eefffd WatchSource:0}: Error finding container 310cf26a01694134856c908235959e67e21a715877a7b7f938a6796ec2eefffd: Status 404 returned error can't find the container with id 310cf26a01694134856c908235959e67e21a715877a7b7f938a6796ec2eefffd
Jul 23 23:39:22.370827 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:22.370790    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1372ee67-66d46b9668-n9mhd" event={"ID":"bebec368-bca0-40ac-9dda-b31c3286b2b7","Type":"ContainerStarted","Data":"310cf26a01694134856c908235959e67e21a715877a7b7f938a6796ec2eefffd"}
Jul 23 23:39:22.577082 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:22.577060    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4"
Jul 23 23:39:22.630591 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:22.630558    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96fceab3-3092-4d94-bcd3-df960292f3a8-istio-envoy\") pod \"96fceab3-3092-4d94-bcd3-df960292f3a8\" (UID: \"96fceab3-3092-4d94-bcd3-df960292f3a8\") "
Jul 23 23:39:22.630772 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:22.630605    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96fceab3-3092-4d94-bcd3-df960292f3a8-istio-token\") pod \"96fceab3-3092-4d94-bcd3-df960292f3a8\" (UID: \"96fceab3-3092-4d94-bcd3-df960292f3a8\") "
Jul 23 23:39:22.630772 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:22.630634    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96fceab3-3092-4d94-bcd3-df960292f3a8-workload-socket\") pod \"96fceab3-3092-4d94-bcd3-df960292f3a8\" (UID: \"96fceab3-3092-4d94-bcd3-df960292f3a8\") "
Jul 23 23:39:22.630772 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:22.630680    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96fceab3-3092-4d94-bcd3-df960292f3a8-istiod-ca-cert\") pod \"96fceab3-3092-4d94-bcd3-df960292f3a8\" (UID: \"96fceab3-3092-4d94-bcd3-df960292f3a8\") "
Jul 23 23:39:22.630931 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:22.630832    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96fceab3-3092-4d94-bcd3-df960292f3a8-credential-socket\") pod \"96fceab3-3092-4d94-bcd3-df960292f3a8\" (UID: \"96fceab3-3092-4d94-bcd3-df960292f3a8\") "
Jul 23 23:39:22.630931 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:22.630909    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96fceab3-3092-4d94-bcd3-df960292f3a8-istio-data\") pod \"96fceab3-3092-4d94-bcd3-df960292f3a8\" (UID: \"96fceab3-3092-4d94-bcd3-df960292f3a8\") "
Jul 23 23:39:22.631035 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:22.630934    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xr56s\" (UniqueName: \"kubernetes.io/projected/96fceab3-3092-4d94-bcd3-df960292f3a8-kube-api-access-xr56s\") pod \"96fceab3-3092-4d94-bcd3-df960292f3a8\" (UID: \"96fceab3-3092-4d94-bcd3-df960292f3a8\") "
Jul 23 23:39:22.631035 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:22.630988    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96fceab3-3092-4d94-bcd3-df960292f3a8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "96fceab3-3092-4d94-bcd3-df960292f3a8" (UID: "96fceab3-3092-4d94-bcd3-df960292f3a8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:39:22.631035 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:22.631001    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96fceab3-3092-4d94-bcd3-df960292f3a8-workload-certs\") pod \"96fceab3-3092-4d94-bcd3-df960292f3a8\" (UID: \"96fceab3-3092-4d94-bcd3-df960292f3a8\") "
Jul 23 23:39:22.631186 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:22.631074    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96fceab3-3092-4d94-bcd3-df960292f3a8-istio-podinfo\") pod \"96fceab3-3092-4d94-bcd3-df960292f3a8\" (UID: \"96fceab3-3092-4d94-bcd3-df960292f3a8\") "
Jul 23 23:39:22.631186 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:22.631088    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96fceab3-3092-4d94-bcd3-df960292f3a8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "96fceab3-3092-4d94-bcd3-df960292f3a8" (UID: "96fceab3-3092-4d94-bcd3-df960292f3a8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:39:22.631372 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:22.631322    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96fceab3-3092-4d94-bcd3-df960292f3a8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "96fceab3-3092-4d94-bcd3-df960292f3a8" (UID: "96fceab3-3092-4d94-bcd3-df960292f3a8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:39:22.631458 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:22.631392    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96fceab3-3092-4d94-bcd3-df960292f3a8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "96fceab3-3092-4d94-bcd3-df960292f3a8" (UID: "96fceab3-3092-4d94-bcd3-df960292f3a8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:39:22.631508 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:22.631458    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96fceab3-3092-4d94-bcd3-df960292f3a8-workload-socket\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:39:22.631508 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:22.631479    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96fceab3-3092-4d94-bcd3-df960292f3a8-credential-socket\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:39:22.631508 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:22.631495    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96fceab3-3092-4d94-bcd3-df960292f3a8-workload-certs\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:39:22.632010 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:22.631977    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96fceab3-3092-4d94-bcd3-df960292f3a8-istio-data" (OuterVolumeSpecName: "istio-data") pod "96fceab3-3092-4d94-bcd3-df960292f3a8" (UID: "96fceab3-3092-4d94-bcd3-df960292f3a8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:39:22.633293 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:22.633260    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96fceab3-3092-4d94-bcd3-df960292f3a8-kube-api-access-xr56s" (OuterVolumeSpecName: "kube-api-access-xr56s") pod "96fceab3-3092-4d94-bcd3-df960292f3a8" (UID: "96fceab3-3092-4d94-bcd3-df960292f3a8"). InnerVolumeSpecName "kube-api-access-xr56s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:39:22.633376 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:22.633308    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/96fceab3-3092-4d94-bcd3-df960292f3a8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "96fceab3-3092-4d94-bcd3-df960292f3a8" (UID: "96fceab3-3092-4d94-bcd3-df960292f3a8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 23:39:22.633423 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:22.633381    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96fceab3-3092-4d94-bcd3-df960292f3a8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "96fceab3-3092-4d94-bcd3-df960292f3a8" (UID: "96fceab3-3092-4d94-bcd3-df960292f3a8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:39:22.633582 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:22.633567    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96fceab3-3092-4d94-bcd3-df960292f3a8-istio-token" (OuterVolumeSpecName: "istio-token") pod "96fceab3-3092-4d94-bcd3-df960292f3a8" (UID: "96fceab3-3092-4d94-bcd3-df960292f3a8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:39:22.732002 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:22.731966    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96fceab3-3092-4d94-bcd3-df960292f3a8-istio-envoy\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:39:22.732002 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:22.731998    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96fceab3-3092-4d94-bcd3-df960292f3a8-istio-token\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:39:22.732002 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:22.732006    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96fceab3-3092-4d94-bcd3-df960292f3a8-istiod-ca-cert\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:39:22.732237 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:22.732014    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96fceab3-3092-4d94-bcd3-df960292f3a8-istio-data\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:39:22.732237 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:22.732026    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xr56s\" (UniqueName: \"kubernetes.io/projected/96fceab3-3092-4d94-bcd3-df960292f3a8-kube-api-access-xr56s\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:39:22.732237 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:22.732036    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96fceab3-3092-4d94-bcd3-df960292f3a8-istio-podinfo\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:39:23.377473 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:23.377432    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1372ee67-66d46b9668-n9mhd" event={"ID":"bebec368-bca0-40ac-9dda-b31c3286b2b7","Type":"ContainerStarted","Data":"917a72634921c1d67984f3086e9d9c2b09026837399267f1324a3d8f19a52860"}
Jul 23 23:39:23.377955 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:23.377538    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1372ee67-66d46b9668-n9mhd"
Jul 23 23:39:23.379246 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:23.379222    2578 generic.go:356] "Generic (PLEG): container finished" podID="96fceab3-3092-4d94-bcd3-df960292f3a8" containerID="4b267ea71d2dc5d16eec3ef9a67acd1b4612d3673930c2ecfbb33252bb5d0f7c" exitCode=0
Jul 23 23:39:23.379367 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:23.379287    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4"
Jul 23 23:39:23.379367 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:23.379296    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4" event={"ID":"96fceab3-3092-4d94-bcd3-df960292f3a8","Type":"ContainerDied","Data":"4b267ea71d2dc5d16eec3ef9a67acd1b4612d3673930c2ecfbb33252bb5d0f7c"}
Jul 23 23:39:23.379367 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:23.379340    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4" event={"ID":"96fceab3-3092-4d94-bcd3-df960292f3a8","Type":"ContainerDied","Data":"216143853ce38bbbe3b700accdff17bacf0be283466354354ad612106e0b5196"}
Jul 23 23:39:23.379367 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:23.379361    2578 scope.go:117] "RemoveContainer" containerID="4b267ea71d2dc5d16eec3ef9a67acd1b4612d3673930c2ecfbb33252bb5d0f7c"
Jul 23 23:39:23.387977 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:23.387957    2578 scope.go:117] "RemoveContainer" containerID="9fc541d7a2078b5d85533fddc02dddf0bfb95194821bf70bcc2c704027168926"
Jul 23 23:39:23.393020 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:23.392977    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1372ee67-66d46b9668-n9mhd" podStartSLOduration=3.080985074 podStartE2EDuration="3.392963436s" totalImagesPullingTime="311.978362ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:39:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:39:23.39237212 +0000 UTC m=+1552.761897780" watchObservedRunningTime="2026-07-23 23:39:23.392963436 +0000 UTC m=+1552.762489096"
Jul 23 23:39:23.397241 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:23.397220    2578 scope.go:117] "RemoveContainer" containerID="4b267ea71d2dc5d16eec3ef9a67acd1b4612d3673930c2ecfbb33252bb5d0f7c"
Jul 23 23:39:23.397514 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:39:23.397495    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b267ea71d2dc5d16eec3ef9a67acd1b4612d3673930c2ecfbb33252bb5d0f7c\": container with ID starting with 4b267ea71d2dc5d16eec3ef9a67acd1b4612d3673930c2ecfbb33252bb5d0f7c not found: ID does not exist" containerID="4b267ea71d2dc5d16eec3ef9a67acd1b4612d3673930c2ecfbb33252bb5d0f7c"
Jul 23 23:39:23.397570 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:23.397523    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b267ea71d2dc5d16eec3ef9a67acd1b4612d3673930c2ecfbb33252bb5d0f7c"} err="failed to get container status \"4b267ea71d2dc5d16eec3ef9a67acd1b4612d3673930c2ecfbb33252bb5d0f7c\": rpc error: code = NotFound desc = could not find container \"4b267ea71d2dc5d16eec3ef9a67acd1b4612d3673930c2ecfbb33252bb5d0f7c\": container with ID starting with 4b267ea71d2dc5d16eec3ef9a67acd1b4612d3673930c2ecfbb33252bb5d0f7c not found: ID does not exist"
Jul 23 23:39:23.397570 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:23.397541    2578 scope.go:117] "RemoveContainer" containerID="9fc541d7a2078b5d85533fddc02dddf0bfb95194821bf70bcc2c704027168926"
Jul 23 23:39:23.397801 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:39:23.397781    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fc541d7a2078b5d85533fddc02dddf0bfb95194821bf70bcc2c704027168926\": container with ID starting with 9fc541d7a2078b5d85533fddc02dddf0bfb95194821bf70bcc2c704027168926 not found: ID does not exist" containerID="9fc541d7a2078b5d85533fddc02dddf0bfb95194821bf70bcc2c704027168926"
Jul 23 23:39:23.397848 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:23.397809    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fc541d7a2078b5d85533fddc02dddf0bfb95194821bf70bcc2c704027168926"} err="failed to get container status \"9fc541d7a2078b5d85533fddc02dddf0bfb95194821bf70bcc2c704027168926\": rpc error: code = NotFound desc = could not find container \"9fc541d7a2078b5d85533fddc02dddf0bfb95194821bf70bcc2c704027168926\": container with ID starting with 9fc541d7a2078b5d85533fddc02dddf0bfb95194821bf70bcc2c704027168926 not found: ID does not exist"
Jul 23 23:39:23.406675 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:23.406583    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4"]
Jul 23 23:39:23.408536 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:23.408512    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-8f14c64d-openshift-default-cdf5dc7d8-ls6x4"]
Jul 23 23:39:25.237851 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:25.237816    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96fceab3-3092-4d94-bcd3-df960292f3a8" path="/var/lib/kubelet/pods/96fceab3-3092-4d94-bcd3-df960292f3a8/volumes"
Jul 23 23:39:29.395579 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:29.395550    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1372ee67-66d46b9668-n9mhd"
Jul 23 23:39:34.431022 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:34.430988    2578 generic.go:356] "Generic (PLEG): container finished" podID="6738398f-fc24-43b6-b213-21b1cf59eb01" containerID="beb6690c86789c43464f5b3483bca1689d2d272e6e01dc2731a3821daef6ec8e" exitCode=0
Jul 23 23:39:34.431439 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:34.431062    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh" event={"ID":"6738398f-fc24-43b6-b213-21b1cf59eb01","Type":"ContainerDied","Data":"beb6690c86789c43464f5b3483bca1689d2d272e6e01dc2731a3821daef6ec8e"}
Jul 23 23:39:34.431564 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:34.431546    2578 scope.go:117] "RemoveContainer" containerID="beb6690c86789c43464f5b3483bca1689d2d272e6e01dc2731a3821daef6ec8e"
Jul 23 23:39:34.432554 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:34.432529    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:39:34.432616 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:34.432597    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:39:34.432668 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:34.432630    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:39:35.361331 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:35.361293    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"
Jul 23 23:39:35.437324 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:35.437295    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh" event={"ID":"6738398f-fc24-43b6-b213-21b1cf59eb01","Type":"ContainerStarted","Data":"e774dd6cb8b12bcd4cf7b503f08d313f36d86228e59d69809232a8913c917733"}
Jul 23 23:39:35.722202 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:35.722169    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"
Jul 23 23:39:35.723492 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:35.723465    2578 patch_prober.go:28] interesting pod/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Jul 23 23:39:35.723609 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:35.723514    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh" podUID="6738398f-fc24-43b6-b213-21b1cf59eb01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Jul 23 23:39:36.722280 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:36.722239    2578 patch_prober.go:28] interesting pod/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Jul 23 23:39:36.722696 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:36.722298    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh" podUID="6738398f-fc24-43b6-b213-21b1cf59eb01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Jul 23 23:39:37.722057 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:37.722023    2578 patch_prober.go:28] interesting pod/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Jul 23 23:39:37.722240 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:37.722079    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh" podUID="6738398f-fc24-43b6-b213-21b1cf59eb01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Jul 23 23:39:38.722732 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:38.722695    2578 patch_prober.go:28] interesting pod/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Jul 23 23:39:38.723106 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:38.722765    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh" podUID="6738398f-fc24-43b6-b213-21b1cf59eb01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Jul 23 23:39:39.726576 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:39.726544    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"
Jul 23 23:39:39.726995 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:39.726819    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"
Jul 23 23:39:39.727585 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:39.727569    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"
Jul 23 23:39:56.240406 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:56.240355    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-1372ee67-66d46b9668-n9mhd"]
Jul 23 23:39:56.240896 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:56.240611    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1372ee67-66d46b9668-n9mhd" podUID="bebec368-bca0-40ac-9dda-b31c3286b2b7" containerName="maas-api" containerID="cri-o://917a72634921c1d67984f3086e9d9c2b09026837399267f1324a3d8f19a52860" gracePeriod=30
Jul 23 23:39:56.497291 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:56.497231    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1372ee67-66d46b9668-n9mhd"
Jul 23 23:39:56.550706 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:56.550630    2578 generic.go:356] "Generic (PLEG): container finished" podID="bebec368-bca0-40ac-9dda-b31c3286b2b7" containerID="917a72634921c1d67984f3086e9d9c2b09026837399267f1324a3d8f19a52860" exitCode=0
Jul 23 23:39:56.550706 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:56.550686    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1372ee67-66d46b9668-n9mhd" event={"ID":"bebec368-bca0-40ac-9dda-b31c3286b2b7","Type":"ContainerDied","Data":"917a72634921c1d67984f3086e9d9c2b09026837399267f1324a3d8f19a52860"}
Jul 23 23:39:56.550909 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:56.550711    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1372ee67-66d46b9668-n9mhd"
Jul 23 23:39:56.550909 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:56.550756    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1372ee67-66d46b9668-n9mhd" event={"ID":"bebec368-bca0-40ac-9dda-b31c3286b2b7","Type":"ContainerDied","Data":"310cf26a01694134856c908235959e67e21a715877a7b7f938a6796ec2eefffd"}
Jul 23 23:39:56.550909 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:56.550779    2578 scope.go:117] "RemoveContainer" containerID="917a72634921c1d67984f3086e9d9c2b09026837399267f1324a3d8f19a52860"
Jul 23 23:39:56.559277 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:56.559259    2578 scope.go:117] "RemoveContainer" containerID="917a72634921c1d67984f3086e9d9c2b09026837399267f1324a3d8f19a52860"
Jul 23 23:39:56.559361 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:56.559291    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bebec368-bca0-40ac-9dda-b31c3286b2b7-maas-api-tls\") pod \"bebec368-bca0-40ac-9dda-b31c3286b2b7\" (UID: \"bebec368-bca0-40ac-9dda-b31c3286b2b7\") "
Jul 23 23:39:56.559361 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:56.559327    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9tbtp\" (UniqueName: \"kubernetes.io/projected/bebec368-bca0-40ac-9dda-b31c3286b2b7-kube-api-access-9tbtp\") pod \"bebec368-bca0-40ac-9dda-b31c3286b2b7\" (UID: \"bebec368-bca0-40ac-9dda-b31c3286b2b7\") "
Jul 23 23:39:56.559477 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:56.559409    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bebec368-bca0-40ac-9dda-b31c3286b2b7-openshift-service-ca\") pod \"bebec368-bca0-40ac-9dda-b31c3286b2b7\" (UID: \"bebec368-bca0-40ac-9dda-b31c3286b2b7\") "
Jul 23 23:39:56.559609 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:39:56.559576    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"917a72634921c1d67984f3086e9d9c2b09026837399267f1324a3d8f19a52860\": container with ID starting with 917a72634921c1d67984f3086e9d9c2b09026837399267f1324a3d8f19a52860 not found: ID does not exist" containerID="917a72634921c1d67984f3086e9d9c2b09026837399267f1324a3d8f19a52860"
Jul 23 23:39:56.559744 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:56.559619    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"917a72634921c1d67984f3086e9d9c2b09026837399267f1324a3d8f19a52860"} err="failed to get container status \"917a72634921c1d67984f3086e9d9c2b09026837399267f1324a3d8f19a52860\": rpc error: code = NotFound desc = could not find container \"917a72634921c1d67984f3086e9d9c2b09026837399267f1324a3d8f19a52860\": container with ID starting with 917a72634921c1d67984f3086e9d9c2b09026837399267f1324a3d8f19a52860 not found: ID does not exist"
Jul 23 23:39:56.559814 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:56.559796    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bebec368-bca0-40ac-9dda-b31c3286b2b7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bebec368-bca0-40ac-9dda-b31c3286b2b7" (UID: "bebec368-bca0-40ac-9dda-b31c3286b2b7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:39:56.561350 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:56.561329    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bebec368-bca0-40ac-9dda-b31c3286b2b7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bebec368-bca0-40ac-9dda-b31c3286b2b7" (UID: "bebec368-bca0-40ac-9dda-b31c3286b2b7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 23:39:56.561873 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:56.561850    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bebec368-bca0-40ac-9dda-b31c3286b2b7-kube-api-access-9tbtp" (OuterVolumeSpecName: "kube-api-access-9tbtp") pod "bebec368-bca0-40ac-9dda-b31c3286b2b7" (UID: "bebec368-bca0-40ac-9dda-b31c3286b2b7"). InnerVolumeSpecName "kube-api-access-9tbtp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:39:56.660474 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:56.660444    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bebec368-bca0-40ac-9dda-b31c3286b2b7-maas-api-tls\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:39:56.660474 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:56.660473    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9tbtp\" (UniqueName: \"kubernetes.io/projected/bebec368-bca0-40ac-9dda-b31c3286b2b7-kube-api-access-9tbtp\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:39:56.660474 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:56.660482    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bebec368-bca0-40ac-9dda-b31c3286b2b7-openshift-service-ca\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:39:56.873266 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:56.873234    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-1372ee67-66d46b9668-n9mhd"]
Jul 23 23:39:56.876933 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:56.876894    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-1372ee67-66d46b9668-n9mhd"]
Jul 23 23:39:57.235781 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:39:57.235741    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bebec368-bca0-40ac-9dda-b31c3286b2b7" path="/var/lib/kubelet/pods/bebec368-bca0-40ac-9dda-b31c3286b2b7/volumes"
Jul 23 23:40:01.086242 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:01.086208    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"]
Jul 23 23:40:01.086637 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:01.086466    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh" podUID="6738398f-fc24-43b6-b213-21b1cf59eb01" containerName="istio-proxy" containerID="cri-o://e774dd6cb8b12bcd4cf7b503f08d313f36d86228e59d69809232a8913c917733" gracePeriod=30
Jul 23 23:40:01.909622 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:01.909579    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm"]
Jul 23 23:40:01.910051 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:01.910032    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96fceab3-3092-4d94-bcd3-df960292f3a8" containerName="istio-proxy"
Jul 23 23:40:01.910142 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:01.910053    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="96fceab3-3092-4d94-bcd3-df960292f3a8" containerName="istio-proxy"
Jul 23 23:40:01.910142 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:01.910080    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bebec368-bca0-40ac-9dda-b31c3286b2b7" containerName="maas-api"
Jul 23 23:40:01.910142 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:01.910089    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="bebec368-bca0-40ac-9dda-b31c3286b2b7" containerName="maas-api"
Jul 23 23:40:01.910284 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:01.910187    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="96fceab3-3092-4d94-bcd3-df960292f3a8" containerName="istio-proxy"
Jul 23 23:40:01.910284 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:01.910205    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="bebec368-bca0-40ac-9dda-b31c3286b2b7" containerName="maas-api"
Jul 23 23:40:01.912390 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:01.912370    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm"
Jul 23 23:40:01.915123 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:01.915105    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-17320d85-openshift-default-dockercfg-xhjmc\""
Jul 23 23:40:01.921980 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:01.921953    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm"]
Jul 23 23:40:02.009753 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:02.009714    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xn8p7\" (UniqueName: \"kubernetes.io/projected/900d43b9-392b-4782-8e17-811c52632bdf-kube-api-access-xn8p7\") pod \"e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm\" (UID: \"900d43b9-392b-4782-8e17-811c52632bdf\") " pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm"
Jul 23 23:40:02.009753 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:02.009758    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/900d43b9-392b-4782-8e17-811c52632bdf-istiod-ca-cert\") pod \"e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm\" (UID: \"900d43b9-392b-4782-8e17-811c52632bdf\") " pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm"
Jul 23 23:40:02.009975 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:02.009798    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/900d43b9-392b-4782-8e17-811c52632bdf-istio-envoy\") pod \"e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm\" (UID: \"900d43b9-392b-4782-8e17-811c52632bdf\") " pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm"
Jul 23 23:40:02.009975 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:02.009849    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/900d43b9-392b-4782-8e17-811c52632bdf-istio-data\") pod \"e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm\" (UID: \"900d43b9-392b-4782-8e17-811c52632bdf\") " pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm"
Jul 23 23:40:02.009975 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:02.009921    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/900d43b9-392b-4782-8e17-811c52632bdf-istio-podinfo\") pod \"e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm\" (UID: \"900d43b9-392b-4782-8e17-811c52632bdf\") " pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm"
Jul 23 23:40:02.009975 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:02.009940    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/900d43b9-392b-4782-8e17-811c52632bdf-workload-certs\") pod \"e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm\" (UID: \"900d43b9-392b-4782-8e17-811c52632bdf\") " pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm"
Jul 23 23:40:02.009975 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:02.009962    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/900d43b9-392b-4782-8e17-811c52632bdf-istio-token\") pod \"e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm\" (UID: \"900d43b9-392b-4782-8e17-811c52632bdf\") " pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm"
Jul 23 23:40:02.010152 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:02.009982    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/900d43b9-392b-4782-8e17-811c52632bdf-workload-socket\") pod \"e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm\" (UID: \"900d43b9-392b-4782-8e17-811c52632bdf\") " pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm"
Jul 23 23:40:02.010152 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:02.009999    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/900d43b9-392b-4782-8e17-811c52632bdf-credential-socket\") pod \"e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm\" (UID: \"900d43b9-392b-4782-8e17-811c52632bdf\") " pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm"
Jul 23 23:40:02.111403 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:02.111357    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/900d43b9-392b-4782-8e17-811c52632bdf-istio-data\") pod \"e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm\" (UID: \"900d43b9-392b-4782-8e17-811c52632bdf\") " pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm"
Jul 23 23:40:02.111894 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:02.111482    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/900d43b9-392b-4782-8e17-811c52632bdf-istio-podinfo\") pod \"e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm\" (UID: \"900d43b9-392b-4782-8e17-811c52632bdf\") " pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm"
Jul 23 23:40:02.111894 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:02.111533    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/900d43b9-392b-4782-8e17-811c52632bdf-workload-certs\") pod \"e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm\" (UID: \"900d43b9-392b-4782-8e17-811c52632bdf\") " pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm"
Jul 23 23:40:02.111894 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:02.111560    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/900d43b9-392b-4782-8e17-811c52632bdf-istio-token\") pod \"e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm\" (UID: \"900d43b9-392b-4782-8e17-811c52632bdf\") " pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm"
Jul 23 23:40:02.111894 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:02.111589    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/900d43b9-392b-4782-8e17-811c52632bdf-workload-socket\") pod \"e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm\" (UID: \"900d43b9-392b-4782-8e17-811c52632bdf\") " pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm"
Jul 23 23:40:02.111894 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:02.111613    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/900d43b9-392b-4782-8e17-811c52632bdf-credential-socket\") pod \"e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm\" (UID: \"900d43b9-392b-4782-8e17-811c52632bdf\") " pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm"
Jul 23 23:40:02.111894 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:02.111688    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xn8p7\" (UniqueName: \"kubernetes.io/projected/900d43b9-392b-4782-8e17-811c52632bdf-kube-api-access-xn8p7\") pod \"e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm\" (UID: \"900d43b9-392b-4782-8e17-811c52632bdf\") " pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm"
Jul 23 23:40:02.111894 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:02.111728    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/900d43b9-392b-4782-8e17-811c52632bdf-istiod-ca-cert\") pod \"e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm\" (UID: \"900d43b9-392b-4782-8e17-811c52632bdf\") " pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm"
Jul 23 23:40:02.111894 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:02.111777    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/900d43b9-392b-4782-8e17-811c52632bdf-istio-envoy\") pod \"e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm\" (UID: \"900d43b9-392b-4782-8e17-811c52632bdf\") " pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm"
Jul 23 23:40:02.112298 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:02.111979    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/900d43b9-392b-4782-8e17-811c52632bdf-istio-data\") pod \"e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm\" (UID: \"900d43b9-392b-4782-8e17-811c52632bdf\") " pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm"
Jul 23 23:40:02.112298 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:02.112220    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/900d43b9-392b-4782-8e17-811c52632bdf-workload-socket\") pod \"e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm\" (UID: \"900d43b9-392b-4782-8e17-811c52632bdf\") " pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm"
Jul 23 23:40:02.112399 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:02.112380    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/900d43b9-392b-4782-8e17-811c52632bdf-credential-socket\") pod \"e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm\" (UID: \"900d43b9-392b-4782-8e17-811c52632bdf\") " pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm"
Jul 23 23:40:02.112567 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:02.112522    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/900d43b9-392b-4782-8e17-811c52632bdf-workload-certs\") pod \"e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm\" (UID: \"900d43b9-392b-4782-8e17-811c52632bdf\") " pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm"
Jul 23 23:40:02.112884 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:02.112643    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/900d43b9-392b-4782-8e17-811c52632bdf-istiod-ca-cert\") pod \"e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm\" (UID: \"900d43b9-392b-4782-8e17-811c52632bdf\") " pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm"
Jul 23 23:40:02.114499 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:02.114475    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/900d43b9-392b-4782-8e17-811c52632bdf-istio-envoy\") pod \"e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm\" (UID: \"900d43b9-392b-4782-8e17-811c52632bdf\") " pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm"
Jul 23 23:40:02.115237 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:02.115212    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/900d43b9-392b-4782-8e17-811c52632bdf-istio-podinfo\") pod \"e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm\" (UID: \"900d43b9-392b-4782-8e17-811c52632bdf\") " pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm"
Jul 23 23:40:02.120084 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:02.120059    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/900d43b9-392b-4782-8e17-811c52632bdf-istio-token\") pod \"e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm\" (UID: \"900d43b9-392b-4782-8e17-811c52632bdf\") " pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm"
Jul 23 23:40:02.120174 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:02.120102    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xn8p7\" (UniqueName: \"kubernetes.io/projected/900d43b9-392b-4782-8e17-811c52632bdf-kube-api-access-xn8p7\") pod \"e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm\" (UID: \"900d43b9-392b-4782-8e17-811c52632bdf\") " pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm"
Jul 23 23:40:02.224390 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:02.224351    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm"
Jul 23 23:40:02.359783 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:02.359749    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm"]
Jul 23 23:40:02.362801 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:40:02.362769    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod900d43b9_392b_4782_8e17_811c52632bdf.slice/crio-af774270cca460ae9ff97b12421915ff5df8360c84c26d0026db043d7e18a6cf WatchSource:0}: Error finding container af774270cca460ae9ff97b12421915ff5df8360c84c26d0026db043d7e18a6cf: Status 404 returned error can't find the container with id af774270cca460ae9ff97b12421915ff5df8360c84c26d0026db043d7e18a6cf
Jul 23 23:40:02.365109 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:02.365075    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:40:02.365194 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:02.365167    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:40:02.365250 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:02.365235    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:40:02.582222 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:02.582137    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm" event={"ID":"900d43b9-392b-4782-8e17-811c52632bdf","Type":"ContainerStarted","Data":"92f1d78500b39abe9da1fdb426b8720163949c88975f0ebfa4e3814001168db6"}
Jul 23 23:40:02.582222 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:02.582202    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm" event={"ID":"900d43b9-392b-4782-8e17-811c52632bdf","Type":"ContainerStarted","Data":"af774270cca460ae9ff97b12421915ff5df8360c84c26d0026db043d7e18a6cf"}
Jul 23 23:40:02.602416 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:02.602360    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm" podStartSLOduration=1.6023394180000001 podStartE2EDuration="1.602339418s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:40:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:40:02.599718645 +0000 UTC m=+1591.969244301" watchObservedRunningTime="2026-07-23 23:40:02.602339418 +0000 UTC m=+1591.971865079"
Jul 23 23:40:03.224549 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:03.224503    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm"
Jul 23 23:40:03.229303 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:03.229277    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm"
Jul 23 23:40:03.587368 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:03.587277    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm"
Jul 23 23:40:03.588307 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:03.588288    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm"
Jul 23 23:40:04.144817 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:04.144785    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-17320d85-7d697fb775-xxtnh"]
Jul 23 23:40:04.145231 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:04.145211    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96fceab3-3092-4d94-bcd3-df960292f3a8" containerName="istio-proxy"
Jul 23 23:40:04.145392 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:04.145234    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="96fceab3-3092-4d94-bcd3-df960292f3a8" containerName="istio-proxy"
Jul 23 23:40:04.145392 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:04.145340    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="96fceab3-3092-4d94-bcd3-df960292f3a8" containerName="istio-proxy"
Jul 23 23:40:04.151350 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:04.151328    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-17320d85-7d697fb775-xxtnh"
Jul 23 23:40:04.154231 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:04.154206    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-17320d85-serving-cert\""
Jul 23 23:40:04.158944 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:04.158915    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-17320d85-7d697fb775-xxtnh"]
Jul 23 23:40:04.242940 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:04.242904    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xs4mn\" (UniqueName: \"kubernetes.io/projected/30cb980f-814e-4155-b4ef-6caa551dd387-kube-api-access-xs4mn\") pod \"maas-api-e2e-ait-17320d85-7d697fb775-xxtnh\" (UID: \"30cb980f-814e-4155-b4ef-6caa551dd387\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-17320d85-7d697fb775-xxtnh"
Jul 23 23:40:04.243390 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:04.242981    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30cb980f-814e-4155-b4ef-6caa551dd387-maas-api-tls\") pod \"maas-api-e2e-ait-17320d85-7d697fb775-xxtnh\" (UID: \"30cb980f-814e-4155-b4ef-6caa551dd387\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-17320d85-7d697fb775-xxtnh"
Jul 23 23:40:04.243390 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:04.243061    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30cb980f-814e-4155-b4ef-6caa551dd387-openshift-service-ca\") pod \"maas-api-e2e-ait-17320d85-7d697fb775-xxtnh\" (UID: \"30cb980f-814e-4155-b4ef-6caa551dd387\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-17320d85-7d697fb775-xxtnh"
Jul 23 23:40:04.343850 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:04.343810    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30cb980f-814e-4155-b4ef-6caa551dd387-openshift-service-ca\") pod \"maas-api-e2e-ait-17320d85-7d697fb775-xxtnh\" (UID: \"30cb980f-814e-4155-b4ef-6caa551dd387\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-17320d85-7d697fb775-xxtnh"
Jul 23 23:40:04.344101 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:04.344079    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xs4mn\" (UniqueName: \"kubernetes.io/projected/30cb980f-814e-4155-b4ef-6caa551dd387-kube-api-access-xs4mn\") pod \"maas-api-e2e-ait-17320d85-7d697fb775-xxtnh\" (UID: \"30cb980f-814e-4155-b4ef-6caa551dd387\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-17320d85-7d697fb775-xxtnh"
Jul 23 23:40:04.344185 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:04.344157    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30cb980f-814e-4155-b4ef-6caa551dd387-maas-api-tls\") pod \"maas-api-e2e-ait-17320d85-7d697fb775-xxtnh\" (UID: \"30cb980f-814e-4155-b4ef-6caa551dd387\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-17320d85-7d697fb775-xxtnh"
Jul 23 23:40:04.344702 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:04.344645    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30cb980f-814e-4155-b4ef-6caa551dd387-openshift-service-ca\") pod \"maas-api-e2e-ait-17320d85-7d697fb775-xxtnh\" (UID: \"30cb980f-814e-4155-b4ef-6caa551dd387\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-17320d85-7d697fb775-xxtnh"
Jul 23 23:40:04.346830 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:04.346801    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30cb980f-814e-4155-b4ef-6caa551dd387-maas-api-tls\") pod \"maas-api-e2e-ait-17320d85-7d697fb775-xxtnh\" (UID: \"30cb980f-814e-4155-b4ef-6caa551dd387\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-17320d85-7d697fb775-xxtnh"
Jul 23 23:40:04.356229 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:04.356203    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xs4mn\" (UniqueName: \"kubernetes.io/projected/30cb980f-814e-4155-b4ef-6caa551dd387-kube-api-access-xs4mn\") pod \"maas-api-e2e-ait-17320d85-7d697fb775-xxtnh\" (UID: \"30cb980f-814e-4155-b4ef-6caa551dd387\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-17320d85-7d697fb775-xxtnh"
Jul 23 23:40:04.464241 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:04.464202    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-17320d85-7d697fb775-xxtnh"
Jul 23 23:40:04.598394 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:04.598369    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-17320d85-7d697fb775-xxtnh"]
Jul 23 23:40:04.600548 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:40:04.600522    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30cb980f_814e_4155_b4ef_6caa551dd387.slice/crio-6dc4c685e39d94da4091700d34be0739a1638a3fb12c316ea0a23c9b36376bda WatchSource:0}: Error finding container 6dc4c685e39d94da4091700d34be0739a1638a3fb12c316ea0a23c9b36376bda: Status 404 returned error can't find the container with id 6dc4c685e39d94da4091700d34be0739a1638a3fb12c316ea0a23c9b36376bda
Jul 23 23:40:05.598663 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:05.598628    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-17320d85-7d697fb775-xxtnh" event={"ID":"30cb980f-814e-4155-b4ef-6caa551dd387","Type":"ContainerStarted","Data":"d4a9499c3931257aa34fcf3df44ac4959e59c8bf83a7d3cb82bb7157f154aaaa"}
Jul 23 23:40:05.599142 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:05.598722    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-17320d85-7d697fb775-xxtnh" event={"ID":"30cb980f-814e-4155-b4ef-6caa551dd387","Type":"ContainerStarted","Data":"6dc4c685e39d94da4091700d34be0739a1638a3fb12c316ea0a23c9b36376bda"}
Jul 23 23:40:05.599142 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:05.598867    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-17320d85-7d697fb775-xxtnh"
Jul 23 23:40:05.617621 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:05.617569    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-17320d85-7d697fb775-xxtnh" podStartSLOduration=1.296298388 podStartE2EDuration="1.617554019s" totalImagesPullingTime="321.255631ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:40:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:40:05.61623344 +0000 UTC m=+1594.985759101" watchObservedRunningTime="2026-07-23 23:40:05.617554019 +0000 UTC m=+1594.987079678"
Jul 23 23:40:06.347560 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:06.347527    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"
Jul 23 23:40:06.467511 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:06.467477    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6738398f-fc24-43b6-b213-21b1cf59eb01-istiod-ca-cert\") pod \"6738398f-fc24-43b6-b213-21b1cf59eb01\" (UID: \"6738398f-fc24-43b6-b213-21b1cf59eb01\") "
Jul 23 23:40:06.467718 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:06.467550    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6738398f-fc24-43b6-b213-21b1cf59eb01-workload-certs\") pod \"6738398f-fc24-43b6-b213-21b1cf59eb01\" (UID: \"6738398f-fc24-43b6-b213-21b1cf59eb01\") "
Jul 23 23:40:06.467718 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:06.467580    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6738398f-fc24-43b6-b213-21b1cf59eb01-istio-podinfo\") pod \"6738398f-fc24-43b6-b213-21b1cf59eb01\" (UID: \"6738398f-fc24-43b6-b213-21b1cf59eb01\") "
Jul 23 23:40:06.467718 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:06.467622    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6738398f-fc24-43b6-b213-21b1cf59eb01-istio-envoy\") pod \"6738398f-fc24-43b6-b213-21b1cf59eb01\" (UID: \"6738398f-fc24-43b6-b213-21b1cf59eb01\") "
Jul 23 23:40:06.467875 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:06.467807    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6738398f-fc24-43b6-b213-21b1cf59eb01-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6738398f-fc24-43b6-b213-21b1cf59eb01" (UID: "6738398f-fc24-43b6-b213-21b1cf59eb01"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:40:06.467875 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:06.467828    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6738398f-fc24-43b6-b213-21b1cf59eb01-istio-data\") pod \"6738398f-fc24-43b6-b213-21b1cf59eb01\" (UID: \"6738398f-fc24-43b6-b213-21b1cf59eb01\") "
Jul 23 23:40:06.467977 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:06.467878    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rxtb7\" (UniqueName: \"kubernetes.io/projected/6738398f-fc24-43b6-b213-21b1cf59eb01-kube-api-access-rxtb7\") pod \"6738398f-fc24-43b6-b213-21b1cf59eb01\" (UID: \"6738398f-fc24-43b6-b213-21b1cf59eb01\") "
Jul 23 23:40:06.467977 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:06.467898    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6738398f-fc24-43b6-b213-21b1cf59eb01-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6738398f-fc24-43b6-b213-21b1cf59eb01" (UID: "6738398f-fc24-43b6-b213-21b1cf59eb01"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:40:06.467977 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:06.467936    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6738398f-fc24-43b6-b213-21b1cf59eb01-credential-socket\") pod \"6738398f-fc24-43b6-b213-21b1cf59eb01\" (UID: \"6738398f-fc24-43b6-b213-21b1cf59eb01\") "
Jul 23 23:40:06.467977 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:06.467964    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6738398f-fc24-43b6-b213-21b1cf59eb01-workload-socket\") pod \"6738398f-fc24-43b6-b213-21b1cf59eb01\" (UID: \"6738398f-fc24-43b6-b213-21b1cf59eb01\") "
Jul 23 23:40:06.468151 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:06.468013    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6738398f-fc24-43b6-b213-21b1cf59eb01-istio-token\") pod \"6738398f-fc24-43b6-b213-21b1cf59eb01\" (UID: \"6738398f-fc24-43b6-b213-21b1cf59eb01\") "
Jul 23 23:40:06.468202 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:06.468172    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6738398f-fc24-43b6-b213-21b1cf59eb01-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6738398f-fc24-43b6-b213-21b1cf59eb01" (UID: "6738398f-fc24-43b6-b213-21b1cf59eb01"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:40:06.468295 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:06.468273    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6738398f-fc24-43b6-b213-21b1cf59eb01-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6738398f-fc24-43b6-b213-21b1cf59eb01" (UID: "6738398f-fc24-43b6-b213-21b1cf59eb01"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:40:06.468460 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:06.468441    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6738398f-fc24-43b6-b213-21b1cf59eb01-workload-certs\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:40:06.468533 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:06.468477    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6738398f-fc24-43b6-b213-21b1cf59eb01-credential-socket\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:40:06.468533 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:06.468490    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6738398f-fc24-43b6-b213-21b1cf59eb01-workload-socket\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:40:06.468533 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:06.468504    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6738398f-fc24-43b6-b213-21b1cf59eb01-istiod-ca-cert\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:40:06.468853 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:06.468761    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6738398f-fc24-43b6-b213-21b1cf59eb01-istio-data" (OuterVolumeSpecName: "istio-data") pod "6738398f-fc24-43b6-b213-21b1cf59eb01" (UID: "6738398f-fc24-43b6-b213-21b1cf59eb01"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:40:06.470258 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:06.470227    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6738398f-fc24-43b6-b213-21b1cf59eb01-kube-api-access-rxtb7" (OuterVolumeSpecName: "kube-api-access-rxtb7") pod "6738398f-fc24-43b6-b213-21b1cf59eb01" (UID: "6738398f-fc24-43b6-b213-21b1cf59eb01"). InnerVolumeSpecName "kube-api-access-rxtb7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:40:06.470370 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:06.470310    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6738398f-fc24-43b6-b213-21b1cf59eb01-istio-token" (OuterVolumeSpecName: "istio-token") pod "6738398f-fc24-43b6-b213-21b1cf59eb01" (UID: "6738398f-fc24-43b6-b213-21b1cf59eb01"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:40:06.470586 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:06.470569    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6738398f-fc24-43b6-b213-21b1cf59eb01-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6738398f-fc24-43b6-b213-21b1cf59eb01" (UID: "6738398f-fc24-43b6-b213-21b1cf59eb01"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:40:06.470640 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:06.470578    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6738398f-fc24-43b6-b213-21b1cf59eb01-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6738398f-fc24-43b6-b213-21b1cf59eb01" (UID: "6738398f-fc24-43b6-b213-21b1cf59eb01"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 23:40:06.569835 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:06.569799    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6738398f-fc24-43b6-b213-21b1cf59eb01-istio-token\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:40:06.569835 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:06.569826    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6738398f-fc24-43b6-b213-21b1cf59eb01-istio-podinfo\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:40:06.569835 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:06.569836    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6738398f-fc24-43b6-b213-21b1cf59eb01-istio-envoy\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:40:06.569835 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:06.569846    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6738398f-fc24-43b6-b213-21b1cf59eb01-istio-data\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:40:06.570086 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:06.569854    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rxtb7\" (UniqueName: \"kubernetes.io/projected/6738398f-fc24-43b6-b213-21b1cf59eb01-kube-api-access-rxtb7\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:40:06.605562 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:06.605531    2578 generic.go:356] "Generic (PLEG): container finished" podID="6738398f-fc24-43b6-b213-21b1cf59eb01" containerID="e774dd6cb8b12bcd4cf7b503f08d313f36d86228e59d69809232a8913c917733" exitCode=0
Jul 23 23:40:06.605973 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:06.605585    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"
Jul 23 23:40:06.605973 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:06.605614    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh" event={"ID":"6738398f-fc24-43b6-b213-21b1cf59eb01","Type":"ContainerDied","Data":"e774dd6cb8b12bcd4cf7b503f08d313f36d86228e59d69809232a8913c917733"}
Jul 23 23:40:06.605973 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:06.605724    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh" event={"ID":"6738398f-fc24-43b6-b213-21b1cf59eb01","Type":"ContainerDied","Data":"13de3e427c58d8e397b45cd524bcf4cc9edf589e0ec045eb0cd75ccef115b6ee"}
Jul 23 23:40:06.605973 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:06.605749    2578 scope.go:117] "RemoveContainer" containerID="e774dd6cb8b12bcd4cf7b503f08d313f36d86228e59d69809232a8913c917733"
Jul 23 23:40:06.615641 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:06.615624    2578 scope.go:117] "RemoveContainer" containerID="beb6690c86789c43464f5b3483bca1689d2d272e6e01dc2731a3821daef6ec8e"
Jul 23 23:40:06.623438 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:06.623423    2578 scope.go:117] "RemoveContainer" containerID="e774dd6cb8b12bcd4cf7b503f08d313f36d86228e59d69809232a8913c917733"
Jul 23 23:40:06.623734 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:40:06.623669    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e774dd6cb8b12bcd4cf7b503f08d313f36d86228e59d69809232a8913c917733\": container with ID starting with e774dd6cb8b12bcd4cf7b503f08d313f36d86228e59d69809232a8913c917733 not found: ID does not exist" containerID="e774dd6cb8b12bcd4cf7b503f08d313f36d86228e59d69809232a8913c917733"
Jul 23 23:40:06.623734 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:06.623701    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e774dd6cb8b12bcd4cf7b503f08d313f36d86228e59d69809232a8913c917733"} err="failed to get container status \"e774dd6cb8b12bcd4cf7b503f08d313f36d86228e59d69809232a8913c917733\": rpc error: code = NotFound desc = could not find container \"e774dd6cb8b12bcd4cf7b503f08d313f36d86228e59d69809232a8913c917733\": container with ID starting with e774dd6cb8b12bcd4cf7b503f08d313f36d86228e59d69809232a8913c917733 not found: ID does not exist"
Jul 23 23:40:06.623734 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:06.623723    2578 scope.go:117] "RemoveContainer" containerID="beb6690c86789c43464f5b3483bca1689d2d272e6e01dc2731a3821daef6ec8e"
Jul 23 23:40:06.624216 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:40:06.624198    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"beb6690c86789c43464f5b3483bca1689d2d272e6e01dc2731a3821daef6ec8e\": container with ID starting with beb6690c86789c43464f5b3483bca1689d2d272e6e01dc2731a3821daef6ec8e not found: ID does not exist" containerID="beb6690c86789c43464f5b3483bca1689d2d272e6e01dc2731a3821daef6ec8e"
Jul 23 23:40:06.624273 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:06.624219    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"beb6690c86789c43464f5b3483bca1689d2d272e6e01dc2731a3821daef6ec8e"} err="failed to get container status \"beb6690c86789c43464f5b3483bca1689d2d272e6e01dc2731a3821daef6ec8e\": rpc error: code = NotFound desc = could not find container \"beb6690c86789c43464f5b3483bca1689d2d272e6e01dc2731a3821daef6ec8e\": container with ID starting with beb6690c86789c43464f5b3483bca1689d2d272e6e01dc2731a3821daef6ec8e not found: ID does not exist"
Jul 23 23:40:06.628188 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:06.628165    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"]
Jul 23 23:40:06.631855 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:06.631836    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-1372ee67-gw-openshift-default-c56bf8d5b-mg8nh"]
Jul 23 23:40:07.237466 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:07.237433    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6738398f-fc24-43b6-b213-21b1cf59eb01" path="/var/lib/kubelet/pods/6738398f-fc24-43b6-b213-21b1cf59eb01/volumes"
Jul 23 23:40:11.611949 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:11.611918    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-17320d85-7d697fb775-xxtnh"
Jul 23 23:40:18.588705 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:18.588668    2578 patch_prober.go:28] interesting pod/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Jul 23 23:40:18.589125 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:18.588738    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm" podUID="900d43b9-392b-4782-8e17-811c52632bdf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Jul 23 23:40:18.665556 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:18.665518    2578 generic.go:356] "Generic (PLEG): container finished" podID="900d43b9-392b-4782-8e17-811c52632bdf" containerID="92f1d78500b39abe9da1fdb426b8720163949c88975f0ebfa4e3814001168db6" exitCode=0
Jul 23 23:40:18.665743 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:18.665593    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm" event={"ID":"900d43b9-392b-4782-8e17-811c52632bdf","Type":"ContainerDied","Data":"92f1d78500b39abe9da1fdb426b8720163949c88975f0ebfa4e3814001168db6"}
Jul 23 23:40:18.666025 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:18.666010    2578 scope.go:117] "RemoveContainer" containerID="92f1d78500b39abe9da1fdb426b8720163949c88975f0ebfa4e3814001168db6"
Jul 23 23:40:18.667100 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:18.667074    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:40:18.667170 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:18.667155    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:40:18.667213 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:18.667194    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:40:19.675966 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:19.675923    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm" event={"ID":"900d43b9-392b-4782-8e17-811c52632bdf","Type":"ContainerStarted","Data":"0e00b447955304f7ca6ae367a974337b5d9be9694945805587a6e29a0cd277b3"}
Jul 23 23:40:20.225029 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:20.224992    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm"
Jul 23 23:40:20.225218 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:20.225194    2578 patch_prober.go:28] interesting pod/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Jul 23 23:40:20.225291 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:20.225267    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm" podUID="900d43b9-392b-4782-8e17-811c52632bdf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Jul 23 23:40:21.225443 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:21.225405    2578 patch_prober.go:28] interesting pod/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Jul 23 23:40:21.225847 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:21.225474    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm" podUID="900d43b9-392b-4782-8e17-811c52632bdf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Jul 23 23:40:22.225741 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:22.225701    2578 patch_prober.go:28] interesting pod/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Jul 23 23:40:22.226148 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:22.225759    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm" podUID="900d43b9-392b-4782-8e17-811c52632bdf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Jul 23 23:40:23.225628 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:23.225594    2578 patch_prober.go:28] interesting pod/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Jul 23 23:40:23.225811 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:23.225676    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm" podUID="900d43b9-392b-4782-8e17-811c52632bdf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Jul 23 23:40:24.229703 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:24.229644    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm"
Jul 23 23:40:24.230081 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:24.229920    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm"
Jul 23 23:40:24.230630 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:24.230613    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm"
Jul 23 23:40:40.259853 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:40.259819    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-17320d85-7d697fb775-xxtnh"]
Jul 23 23:40:40.260386 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:40.260049    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-17320d85-7d697fb775-xxtnh" podUID="30cb980f-814e-4155-b4ef-6caa551dd387" containerName="maas-api" containerID="cri-o://d4a9499c3931257aa34fcf3df44ac4959e59c8bf83a7d3cb82bb7157f154aaaa" gracePeriod=30
Jul 23 23:40:40.517886 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:40.517816    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-17320d85-7d697fb775-xxtnh"
Jul 23 23:40:40.603590 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:40.603549    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30cb980f-814e-4155-b4ef-6caa551dd387-openshift-service-ca\") pod \"30cb980f-814e-4155-b4ef-6caa551dd387\" (UID: \"30cb980f-814e-4155-b4ef-6caa551dd387\") "
Jul 23 23:40:40.603810 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:40.603605    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30cb980f-814e-4155-b4ef-6caa551dd387-maas-api-tls\") pod \"30cb980f-814e-4155-b4ef-6caa551dd387\" (UID: \"30cb980f-814e-4155-b4ef-6caa551dd387\") "
Jul 23 23:40:40.603810 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:40.603680    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xs4mn\" (UniqueName: \"kubernetes.io/projected/30cb980f-814e-4155-b4ef-6caa551dd387-kube-api-access-xs4mn\") pod \"30cb980f-814e-4155-b4ef-6caa551dd387\" (UID: \"30cb980f-814e-4155-b4ef-6caa551dd387\") "
Jul 23 23:40:40.603952 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:40.603927    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30cb980f-814e-4155-b4ef-6caa551dd387-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "30cb980f-814e-4155-b4ef-6caa551dd387" (UID: "30cb980f-814e-4155-b4ef-6caa551dd387"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:40:40.604054 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:40.604041    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30cb980f-814e-4155-b4ef-6caa551dd387-openshift-service-ca\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:40:40.606005 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:40.605979    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30cb980f-814e-4155-b4ef-6caa551dd387-kube-api-access-xs4mn" (OuterVolumeSpecName: "kube-api-access-xs4mn") pod "30cb980f-814e-4155-b4ef-6caa551dd387" (UID: "30cb980f-814e-4155-b4ef-6caa551dd387"). InnerVolumeSpecName "kube-api-access-xs4mn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:40:40.606005 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:40.605982    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30cb980f-814e-4155-b4ef-6caa551dd387-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "30cb980f-814e-4155-b4ef-6caa551dd387" (UID: "30cb980f-814e-4155-b4ef-6caa551dd387"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 23:40:40.705114 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:40.705077    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30cb980f-814e-4155-b4ef-6caa551dd387-maas-api-tls\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:40:40.705114 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:40.705108    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xs4mn\" (UniqueName: \"kubernetes.io/projected/30cb980f-814e-4155-b4ef-6caa551dd387-kube-api-access-xs4mn\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:40:40.781530 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:40.781452    2578 generic.go:356] "Generic (PLEG): container finished" podID="30cb980f-814e-4155-b4ef-6caa551dd387" containerID="d4a9499c3931257aa34fcf3df44ac4959e59c8bf83a7d3cb82bb7157f154aaaa" exitCode=0
Jul 23 23:40:40.781530 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:40.781516    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-17320d85-7d697fb775-xxtnh"
Jul 23 23:40:40.781717 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:40.781536    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-17320d85-7d697fb775-xxtnh" event={"ID":"30cb980f-814e-4155-b4ef-6caa551dd387","Type":"ContainerDied","Data":"d4a9499c3931257aa34fcf3df44ac4959e59c8bf83a7d3cb82bb7157f154aaaa"}
Jul 23 23:40:40.781717 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:40.781596    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-17320d85-7d697fb775-xxtnh" event={"ID":"30cb980f-814e-4155-b4ef-6caa551dd387","Type":"ContainerDied","Data":"6dc4c685e39d94da4091700d34be0739a1638a3fb12c316ea0a23c9b36376bda"}
Jul 23 23:40:40.781717 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:40.781619    2578 scope.go:117] "RemoveContainer" containerID="d4a9499c3931257aa34fcf3df44ac4959e59c8bf83a7d3cb82bb7157f154aaaa"
Jul 23 23:40:40.790185 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:40.790166    2578 scope.go:117] "RemoveContainer" containerID="d4a9499c3931257aa34fcf3df44ac4959e59c8bf83a7d3cb82bb7157f154aaaa"
Jul 23 23:40:40.790509 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:40:40.790488    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4a9499c3931257aa34fcf3df44ac4959e59c8bf83a7d3cb82bb7157f154aaaa\": container with ID starting with d4a9499c3931257aa34fcf3df44ac4959e59c8bf83a7d3cb82bb7157f154aaaa not found: ID does not exist" containerID="d4a9499c3931257aa34fcf3df44ac4959e59c8bf83a7d3cb82bb7157f154aaaa"
Jul 23 23:40:40.790568 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:40.790519    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4a9499c3931257aa34fcf3df44ac4959e59c8bf83a7d3cb82bb7157f154aaaa"} err="failed to get container status \"d4a9499c3931257aa34fcf3df44ac4959e59c8bf83a7d3cb82bb7157f154aaaa\": rpc error: code = NotFound desc = could not find container \"d4a9499c3931257aa34fcf3df44ac4959e59c8bf83a7d3cb82bb7157f154aaaa\": container with ID starting with d4a9499c3931257aa34fcf3df44ac4959e59c8bf83a7d3cb82bb7157f154aaaa not found: ID does not exist"
Jul 23 23:40:40.807557 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:40.807529    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-17320d85-7d697fb775-xxtnh"]
Jul 23 23:40:40.811133 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:40.811105    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-17320d85-7d697fb775-xxtnh"]
Jul 23 23:40:41.236577 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:41.236546    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30cb980f-814e-4155-b4ef-6caa551dd387" path="/var/lib/kubelet/pods/30cb980f-814e-4155-b4ef-6caa551dd387/volumes"
Jul 23 23:40:46.177402 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:46.177361    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm"]
Jul 23 23:40:46.177936 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:46.177635    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm" podUID="900d43b9-392b-4782-8e17-811c52632bdf" containerName="istio-proxy" containerID="cri-o://0e00b447955304f7ca6ae367a974337b5d9be9694945805587a6e29a0cd277b3" gracePeriod=30
Jul 23 23:40:46.937799 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:46.937767    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"]
Jul 23 23:40:46.938198 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:46.938177    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6738398f-fc24-43b6-b213-21b1cf59eb01" containerName="istio-proxy"
Jul 23 23:40:46.938198 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:46.938195    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6738398f-fc24-43b6-b213-21b1cf59eb01" containerName="istio-proxy"
Jul 23 23:40:46.938330 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:46.938220    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30cb980f-814e-4155-b4ef-6caa551dd387" containerName="maas-api"
Jul 23 23:40:46.938330 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:46.938226    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="30cb980f-814e-4155-b4ef-6caa551dd387" containerName="maas-api"
Jul 23 23:40:46.938330 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:46.938284    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="30cb980f-814e-4155-b4ef-6caa551dd387" containerName="maas-api"
Jul 23 23:40:46.938330 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:46.938297    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6738398f-fc24-43b6-b213-21b1cf59eb01" containerName="istio-proxy"
Jul 23 23:40:46.941269 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:46.941251    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"
Jul 23 23:40:46.944072 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:46.944051    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-7cb9a6a8-openshift-default-dockercfg-5brb7\""
Jul 23 23:40:46.953027 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:46.953002    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"]
Jul 23 23:40:47.068576 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:47.068537    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ntl5\" (UniqueName: \"kubernetes.io/projected/ac020e93-9fb2-4630-908b-d05fbda10ec0-kube-api-access-8ntl5\") pod \"e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq\" (UID: \"ac020e93-9fb2-4630-908b-d05fbda10ec0\") " pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"
Jul 23 23:40:47.068782 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:47.068673    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac020e93-9fb2-4630-908b-d05fbda10ec0-workload-certs\") pod \"e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq\" (UID: \"ac020e93-9fb2-4630-908b-d05fbda10ec0\") " pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"
Jul 23 23:40:47.068782 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:47.068712    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac020e93-9fb2-4630-908b-d05fbda10ec0-istio-envoy\") pod \"e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq\" (UID: \"ac020e93-9fb2-4630-908b-d05fbda10ec0\") " pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"
Jul 23 23:40:47.068782 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:47.068732    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac020e93-9fb2-4630-908b-d05fbda10ec0-istiod-ca-cert\") pod \"e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq\" (UID: \"ac020e93-9fb2-4630-908b-d05fbda10ec0\") " pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"
Jul 23 23:40:47.068782 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:47.068751    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac020e93-9fb2-4630-908b-d05fbda10ec0-workload-socket\") pod \"e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq\" (UID: \"ac020e93-9fb2-4630-908b-d05fbda10ec0\") " pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"
Jul 23 23:40:47.068997 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:47.068798    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac020e93-9fb2-4630-908b-d05fbda10ec0-istio-podinfo\") pod \"e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq\" (UID: \"ac020e93-9fb2-4630-908b-d05fbda10ec0\") " pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"
Jul 23 23:40:47.068997 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:47.068826    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac020e93-9fb2-4630-908b-d05fbda10ec0-istio-data\") pod \"e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq\" (UID: \"ac020e93-9fb2-4630-908b-d05fbda10ec0\") " pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"
Jul 23 23:40:47.068997 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:47.068849    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac020e93-9fb2-4630-908b-d05fbda10ec0-istio-token\") pod \"e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq\" (UID: \"ac020e93-9fb2-4630-908b-d05fbda10ec0\") " pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"
Jul 23 23:40:47.068997 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:47.068894    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac020e93-9fb2-4630-908b-d05fbda10ec0-credential-socket\") pod \"e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq\" (UID: \"ac020e93-9fb2-4630-908b-d05fbda10ec0\") " pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"
Jul 23 23:40:47.169704 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:47.169621    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac020e93-9fb2-4630-908b-d05fbda10ec0-istio-podinfo\") pod \"e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq\" (UID: \"ac020e93-9fb2-4630-908b-d05fbda10ec0\") " pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"
Jul 23 23:40:47.169704 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:47.169709    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac020e93-9fb2-4630-908b-d05fbda10ec0-istio-data\") pod \"e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq\" (UID: \"ac020e93-9fb2-4630-908b-d05fbda10ec0\") " pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"
Jul 23 23:40:47.169943 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:47.169736    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac020e93-9fb2-4630-908b-d05fbda10ec0-istio-token\") pod \"e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq\" (UID: \"ac020e93-9fb2-4630-908b-d05fbda10ec0\") " pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"
Jul 23 23:40:47.169943 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:47.169774    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac020e93-9fb2-4630-908b-d05fbda10ec0-credential-socket\") pod \"e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq\" (UID: \"ac020e93-9fb2-4630-908b-d05fbda10ec0\") " pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"
Jul 23 23:40:47.169943 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:47.169804    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ntl5\" (UniqueName: \"kubernetes.io/projected/ac020e93-9fb2-4630-908b-d05fbda10ec0-kube-api-access-8ntl5\") pod \"e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq\" (UID: \"ac020e93-9fb2-4630-908b-d05fbda10ec0\") " pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"
Jul 23 23:40:47.169943 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:47.169898    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac020e93-9fb2-4630-908b-d05fbda10ec0-workload-certs\") pod \"e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq\" (UID: \"ac020e93-9fb2-4630-908b-d05fbda10ec0\") " pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"
Jul 23 23:40:47.169943 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:47.169925    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac020e93-9fb2-4630-908b-d05fbda10ec0-istio-envoy\") pod \"e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq\" (UID: \"ac020e93-9fb2-4630-908b-d05fbda10ec0\") " pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"
Jul 23 23:40:47.170221 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:47.169951    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac020e93-9fb2-4630-908b-d05fbda10ec0-istiod-ca-cert\") pod \"e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq\" (UID: \"ac020e93-9fb2-4630-908b-d05fbda10ec0\") " pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"
Jul 23 23:40:47.170221 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:47.169975    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac020e93-9fb2-4630-908b-d05fbda10ec0-workload-socket\") pod \"e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq\" (UID: \"ac020e93-9fb2-4630-908b-d05fbda10ec0\") " pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"
Jul 23 23:40:47.170221 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:47.170107    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac020e93-9fb2-4630-908b-d05fbda10ec0-istio-data\") pod \"e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq\" (UID: \"ac020e93-9fb2-4630-908b-d05fbda10ec0\") " pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"
Jul 23 23:40:47.170221 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:47.170178    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac020e93-9fb2-4630-908b-d05fbda10ec0-credential-socket\") pod \"e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq\" (UID: \"ac020e93-9fb2-4630-908b-d05fbda10ec0\") " pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"
Jul 23 23:40:47.170356 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:47.170339    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac020e93-9fb2-4630-908b-d05fbda10ec0-workload-certs\") pod \"e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq\" (UID: \"ac020e93-9fb2-4630-908b-d05fbda10ec0\") " pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"
Jul 23 23:40:47.170437 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:47.170416    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac020e93-9fb2-4630-908b-d05fbda10ec0-workload-socket\") pod \"e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq\" (UID: \"ac020e93-9fb2-4630-908b-d05fbda10ec0\") " pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"
Jul 23 23:40:47.170746 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:47.170728    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac020e93-9fb2-4630-908b-d05fbda10ec0-istiod-ca-cert\") pod \"e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq\" (UID: \"ac020e93-9fb2-4630-908b-d05fbda10ec0\") " pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"
Jul 23 23:40:47.172192 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:47.172173    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac020e93-9fb2-4630-908b-d05fbda10ec0-istio-envoy\") pod \"e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq\" (UID: \"ac020e93-9fb2-4630-908b-d05fbda10ec0\") " pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"
Jul 23 23:40:47.172382 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:47.172366    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac020e93-9fb2-4630-908b-d05fbda10ec0-istio-podinfo\") pod \"e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq\" (UID: \"ac020e93-9fb2-4630-908b-d05fbda10ec0\") " pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"
Jul 23 23:40:47.177990 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:47.177959    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac020e93-9fb2-4630-908b-d05fbda10ec0-istio-token\") pod \"e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq\" (UID: \"ac020e93-9fb2-4630-908b-d05fbda10ec0\") " pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"
Jul 23 23:40:47.178770 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:47.178750    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ntl5\" (UniqueName: \"kubernetes.io/projected/ac020e93-9fb2-4630-908b-d05fbda10ec0-kube-api-access-8ntl5\") pod \"e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq\" (UID: \"ac020e93-9fb2-4630-908b-d05fbda10ec0\") " pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"
Jul 23 23:40:47.252108 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:47.252019    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"
Jul 23 23:40:47.406204 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:47.406177    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"]
Jul 23 23:40:47.408144 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:40:47.408115    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac020e93_9fb2_4630_908b_d05fbda10ec0.slice/crio-539b30985f080a876853f288ff5c66419d1024ff86a33a918ee845ff674a432a WatchSource:0}: Error finding container 539b30985f080a876853f288ff5c66419d1024ff86a33a918ee845ff674a432a: Status 404 returned error can't find the container with id 539b30985f080a876853f288ff5c66419d1024ff86a33a918ee845ff674a432a
Jul 23 23:40:47.410496 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:47.410462    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:40:47.410594 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:47.410530    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:40:47.410594 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:47.410567    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:40:47.816879 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:47.816838    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq" event={"ID":"ac020e93-9fb2-4630-908b-d05fbda10ec0","Type":"ContainerStarted","Data":"7ade4f3356d6be4f4e1bd40e6560717fc2f0310670d66adc7f1976047492c4f7"}
Jul 23 23:40:47.817054 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:47.816901    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq" event={"ID":"ac020e93-9fb2-4630-908b-d05fbda10ec0","Type":"ContainerStarted","Data":"539b30985f080a876853f288ff5c66419d1024ff86a33a918ee845ff674a432a"}
Jul 23 23:40:47.837927 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:47.837872    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq" podStartSLOduration=1.837856108 podStartE2EDuration="1.837856108s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:40:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:40:47.836085223 +0000 UTC m=+1637.205610884" watchObservedRunningTime="2026-07-23 23:40:47.837856108 +0000 UTC m=+1637.207381766"
Jul 23 23:40:48.252910 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:48.252871    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"
Jul 23 23:40:48.257778 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:48.257752    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"
Jul 23 23:40:48.821879 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:48.821839    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"
Jul 23 23:40:48.822958 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:48.822938    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"
Jul 23 23:40:49.243762 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:49.243726    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7cb9a6a8-8584969b75-vs5b5"]
Jul 23 23:40:49.244095 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:49.244083    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6738398f-fc24-43b6-b213-21b1cf59eb01" containerName="istio-proxy"
Jul 23 23:40:49.244095 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:49.244096    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6738398f-fc24-43b6-b213-21b1cf59eb01" containerName="istio-proxy"
Jul 23 23:40:49.244192 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:49.244186    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6738398f-fc24-43b6-b213-21b1cf59eb01" containerName="istio-proxy"
Jul 23 23:40:49.246458 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:49.246441    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7cb9a6a8-8584969b75-vs5b5"
Jul 23 23:40:49.251093 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:49.251069    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-7cb9a6a8-serving-cert\""
Jul 23 23:40:49.285374 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:49.285344    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7cb9a6a8-8584969b75-vs5b5"]
Jul 23 23:40:49.395679 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:49.395621    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac64aeee-7def-41e3-ac9c-d26daed92052-openshift-service-ca\") pod \"maas-api-e2e-derive-7cb9a6a8-8584969b75-vs5b5\" (UID: \"ac64aeee-7def-41e3-ac9c-d26daed92052\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7cb9a6a8-8584969b75-vs5b5"
Jul 23 23:40:49.395902 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:49.395694    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac64aeee-7def-41e3-ac9c-d26daed92052-maas-api-tls\") pod \"maas-api-e2e-derive-7cb9a6a8-8584969b75-vs5b5\" (UID: \"ac64aeee-7def-41e3-ac9c-d26daed92052\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7cb9a6a8-8584969b75-vs5b5"
Jul 23 23:40:49.395902 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:49.395745    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kr57b\" (UniqueName: \"kubernetes.io/projected/ac64aeee-7def-41e3-ac9c-d26daed92052-kube-api-access-kr57b\") pod \"maas-api-e2e-derive-7cb9a6a8-8584969b75-vs5b5\" (UID: \"ac64aeee-7def-41e3-ac9c-d26daed92052\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7cb9a6a8-8584969b75-vs5b5"
Jul 23 23:40:49.496346 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:49.496264    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac64aeee-7def-41e3-ac9c-d26daed92052-openshift-service-ca\") pod \"maas-api-e2e-derive-7cb9a6a8-8584969b75-vs5b5\" (UID: \"ac64aeee-7def-41e3-ac9c-d26daed92052\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7cb9a6a8-8584969b75-vs5b5"
Jul 23 23:40:49.496346 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:49.496299    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac64aeee-7def-41e3-ac9c-d26daed92052-maas-api-tls\") pod \"maas-api-e2e-derive-7cb9a6a8-8584969b75-vs5b5\" (UID: \"ac64aeee-7def-41e3-ac9c-d26daed92052\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7cb9a6a8-8584969b75-vs5b5"
Jul 23 23:40:49.496596 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:49.496361    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kr57b\" (UniqueName: \"kubernetes.io/projected/ac64aeee-7def-41e3-ac9c-d26daed92052-kube-api-access-kr57b\") pod \"maas-api-e2e-derive-7cb9a6a8-8584969b75-vs5b5\" (UID: \"ac64aeee-7def-41e3-ac9c-d26daed92052\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7cb9a6a8-8584969b75-vs5b5"
Jul 23 23:40:49.496596 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:40:49.496482    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-7cb9a6a8-serving-cert: secret "maas-api-e2e-derive-7cb9a6a8-serving-cert" not found
Jul 23 23:40:49.496596 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:40:49.496573    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac64aeee-7def-41e3-ac9c-d26daed92052-maas-api-tls podName:ac64aeee-7def-41e3-ac9c-d26daed92052 nodeName:}" failed. No retries permitted until 2026-07-23 23:40:49.996548885 +0000 UTC m=+1639.366074524 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ac64aeee-7def-41e3-ac9c-d26daed92052-maas-api-tls") pod "maas-api-e2e-derive-7cb9a6a8-8584969b75-vs5b5" (UID: "ac64aeee-7def-41e3-ac9c-d26daed92052") : secret "maas-api-e2e-derive-7cb9a6a8-serving-cert" not found
Jul 23 23:40:49.496958 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:49.496940    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac64aeee-7def-41e3-ac9c-d26daed92052-openshift-service-ca\") pod \"maas-api-e2e-derive-7cb9a6a8-8584969b75-vs5b5\" (UID: \"ac64aeee-7def-41e3-ac9c-d26daed92052\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7cb9a6a8-8584969b75-vs5b5"
Jul 23 23:40:49.509389 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:49.509356    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kr57b\" (UniqueName: \"kubernetes.io/projected/ac64aeee-7def-41e3-ac9c-d26daed92052-kube-api-access-kr57b\") pod \"maas-api-e2e-derive-7cb9a6a8-8584969b75-vs5b5\" (UID: \"ac64aeee-7def-41e3-ac9c-d26daed92052\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7cb9a6a8-8584969b75-vs5b5"
Jul 23 23:40:50.001967 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:50.001923    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac64aeee-7def-41e3-ac9c-d26daed92052-maas-api-tls\") pod \"maas-api-e2e-derive-7cb9a6a8-8584969b75-vs5b5\" (UID: \"ac64aeee-7def-41e3-ac9c-d26daed92052\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7cb9a6a8-8584969b75-vs5b5"
Jul 23 23:40:50.004389 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:50.004364    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac64aeee-7def-41e3-ac9c-d26daed92052-maas-api-tls\") pod \"maas-api-e2e-derive-7cb9a6a8-8584969b75-vs5b5\" (UID: \"ac64aeee-7def-41e3-ac9c-d26daed92052\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7cb9a6a8-8584969b75-vs5b5"
Jul 23 23:40:50.157874 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:50.157830    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7cb9a6a8-8584969b75-vs5b5"
Jul 23 23:40:50.289707 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:50.289668    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7cb9a6a8-8584969b75-vs5b5"]
Jul 23 23:40:50.291748 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:40:50.291720    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac64aeee_7def_41e3_ac9c_d26daed92052.slice/crio-317e66623c68754842ae9560d99db376d35d4ddcd184b86c51f15ee003fe133a WatchSource:0}: Error finding container 317e66623c68754842ae9560d99db376d35d4ddcd184b86c51f15ee003fe133a: Status 404 returned error can't find the container with id 317e66623c68754842ae9560d99db376d35d4ddcd184b86c51f15ee003fe133a
Jul 23 23:40:50.832908 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:50.832862    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7cb9a6a8-8584969b75-vs5b5" event={"ID":"ac64aeee-7def-41e3-ac9c-d26daed92052","Type":"ContainerStarted","Data":"a4c1e34a54510c331a7d263b6c517ab8cbc7e929bf621178fca3dee46f6c577f"}
Jul 23 23:40:50.833108 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:50.832923    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7cb9a6a8-8584969b75-vs5b5" event={"ID":"ac64aeee-7def-41e3-ac9c-d26daed92052","Type":"ContainerStarted","Data":"317e66623c68754842ae9560d99db376d35d4ddcd184b86c51f15ee003fe133a"}
Jul 23 23:40:50.850023 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:50.849973    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7cb9a6a8-8584969b75-vs5b5" podStartSLOduration=1.466855643 podStartE2EDuration="1.849956937s" totalImagesPullingTime="383.101294ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:40:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:40:50.849184238 +0000 UTC m=+1640.218709908" watchObservedRunningTime="2026-07-23 23:40:50.849956937 +0000 UTC m=+1640.219482597"
Jul 23 23:40:51.436047 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:51.435985    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm"
Jul 23 23:40:51.616913 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:51.616880    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/900d43b9-392b-4782-8e17-811c52632bdf-workload-certs\") pod \"900d43b9-392b-4782-8e17-811c52632bdf\" (UID: \"900d43b9-392b-4782-8e17-811c52632bdf\") "
Jul 23 23:40:51.616913 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:51.616932    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/900d43b9-392b-4782-8e17-811c52632bdf-istio-token\") pod \"900d43b9-392b-4782-8e17-811c52632bdf\" (UID: \"900d43b9-392b-4782-8e17-811c52632bdf\") "
Jul 23 23:40:51.617177 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:51.616971    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/900d43b9-392b-4782-8e17-811c52632bdf-istiod-ca-cert\") pod \"900d43b9-392b-4782-8e17-811c52632bdf\" (UID: \"900d43b9-392b-4782-8e17-811c52632bdf\") "
Jul 23 23:40:51.617177 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:51.617002    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/900d43b9-392b-4782-8e17-811c52632bdf-workload-socket\") pod \"900d43b9-392b-4782-8e17-811c52632bdf\" (UID: \"900d43b9-392b-4782-8e17-811c52632bdf\") "
Jul 23 23:40:51.617177 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:51.617054    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/900d43b9-392b-4782-8e17-811c52632bdf-istio-envoy\") pod \"900d43b9-392b-4782-8e17-811c52632bdf\" (UID: \"900d43b9-392b-4782-8e17-811c52632bdf\") "
Jul 23 23:40:51.617177 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:51.617114    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/900d43b9-392b-4782-8e17-811c52632bdf-credential-socket\") pod \"900d43b9-392b-4782-8e17-811c52632bdf\" (UID: \"900d43b9-392b-4782-8e17-811c52632bdf\") "
Jul 23 23:40:51.617177 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:51.617140    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/900d43b9-392b-4782-8e17-811c52632bdf-istio-data\") pod \"900d43b9-392b-4782-8e17-811c52632bdf\" (UID: \"900d43b9-392b-4782-8e17-811c52632bdf\") "
Jul 23 23:40:51.617463 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:51.617193    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/900d43b9-392b-4782-8e17-811c52632bdf-istio-podinfo\") pod \"900d43b9-392b-4782-8e17-811c52632bdf\" (UID: \"900d43b9-392b-4782-8e17-811c52632bdf\") "
Jul 23 23:40:51.617463 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:51.617221    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/900d43b9-392b-4782-8e17-811c52632bdf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "900d43b9-392b-4782-8e17-811c52632bdf" (UID: "900d43b9-392b-4782-8e17-811c52632bdf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:40:51.617463 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:51.617270    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xn8p7\" (UniqueName: \"kubernetes.io/projected/900d43b9-392b-4782-8e17-811c52632bdf-kube-api-access-xn8p7\") pod \"900d43b9-392b-4782-8e17-811c52632bdf\" (UID: \"900d43b9-392b-4782-8e17-811c52632bdf\") "
Jul 23 23:40:51.617463 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:51.617433    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/900d43b9-392b-4782-8e17-811c52632bdf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "900d43b9-392b-4782-8e17-811c52632bdf" (UID: "900d43b9-392b-4782-8e17-811c52632bdf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:40:51.617690 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:51.617530    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/900d43b9-392b-4782-8e17-811c52632bdf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "900d43b9-392b-4782-8e17-811c52632bdf" (UID: "900d43b9-392b-4782-8e17-811c52632bdf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:40:51.617690 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:51.617563    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/900d43b9-392b-4782-8e17-811c52632bdf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "900d43b9-392b-4782-8e17-811c52632bdf" (UID: "900d43b9-392b-4782-8e17-811c52632bdf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:40:51.617788 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:51.617687    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/900d43b9-392b-4782-8e17-811c52632bdf-workload-socket\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:40:51.617788 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:51.617708    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/900d43b9-392b-4782-8e17-811c52632bdf-credential-socket\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:40:51.617788 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:51.617723    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/900d43b9-392b-4782-8e17-811c52632bdf-workload-certs\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:40:51.617788 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:51.617736    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/900d43b9-392b-4782-8e17-811c52632bdf-istiod-ca-cert\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:40:51.618272 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:51.618244    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/900d43b9-392b-4782-8e17-811c52632bdf-istio-data" (OuterVolumeSpecName: "istio-data") pod "900d43b9-392b-4782-8e17-811c52632bdf" (UID: "900d43b9-392b-4782-8e17-811c52632bdf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:40:51.619448 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:51.619423    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/900d43b9-392b-4782-8e17-811c52632bdf-istio-token" (OuterVolumeSpecName: "istio-token") pod "900d43b9-392b-4782-8e17-811c52632bdf" (UID: "900d43b9-392b-4782-8e17-811c52632bdf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:40:51.619696 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:51.619624    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/900d43b9-392b-4782-8e17-811c52632bdf-kube-api-access-xn8p7" (OuterVolumeSpecName: "kube-api-access-xn8p7") pod "900d43b9-392b-4782-8e17-811c52632bdf" (UID: "900d43b9-392b-4782-8e17-811c52632bdf"). InnerVolumeSpecName "kube-api-access-xn8p7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:40:51.619696 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:51.619643    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/900d43b9-392b-4782-8e17-811c52632bdf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "900d43b9-392b-4782-8e17-811c52632bdf" (UID: "900d43b9-392b-4782-8e17-811c52632bdf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:40:51.619823 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:51.619727    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/900d43b9-392b-4782-8e17-811c52632bdf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "900d43b9-392b-4782-8e17-811c52632bdf" (UID: "900d43b9-392b-4782-8e17-811c52632bdf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 23:40:51.718388 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:51.718303    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/900d43b9-392b-4782-8e17-811c52632bdf-istio-envoy\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:40:51.718388 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:51.718334    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/900d43b9-392b-4782-8e17-811c52632bdf-istio-data\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:40:51.718388 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:51.718342    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/900d43b9-392b-4782-8e17-811c52632bdf-istio-podinfo\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:40:51.718388 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:51.718350    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xn8p7\" (UniqueName: \"kubernetes.io/projected/900d43b9-392b-4782-8e17-811c52632bdf-kube-api-access-xn8p7\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:40:51.718388 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:51.718360    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/900d43b9-392b-4782-8e17-811c52632bdf-istio-token\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:40:51.838725 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:51.838684    2578 generic.go:356] "Generic (PLEG): container finished" podID="900d43b9-392b-4782-8e17-811c52632bdf" containerID="0e00b447955304f7ca6ae367a974337b5d9be9694945805587a6e29a0cd277b3" exitCode=0
Jul 23 23:40:51.838862 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:51.838731    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm"
Jul 23 23:40:51.838862 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:51.838759    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm" event={"ID":"900d43b9-392b-4782-8e17-811c52632bdf","Type":"ContainerDied","Data":"0e00b447955304f7ca6ae367a974337b5d9be9694945805587a6e29a0cd277b3"}
Jul 23 23:40:51.838862 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:51.838816    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm" event={"ID":"900d43b9-392b-4782-8e17-811c52632bdf","Type":"ContainerDied","Data":"af774270cca460ae9ff97b12421915ff5df8360c84c26d0026db043d7e18a6cf"}
Jul 23 23:40:51.838862 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:51.838842    2578 scope.go:117] "RemoveContainer" containerID="0e00b447955304f7ca6ae367a974337b5d9be9694945805587a6e29a0cd277b3"
Jul 23 23:40:51.839107 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:51.839086    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7cb9a6a8-8584969b75-vs5b5"
Jul 23 23:40:51.851834 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:51.851817    2578 scope.go:117] "RemoveContainer" containerID="92f1d78500b39abe9da1fdb426b8720163949c88975f0ebfa4e3814001168db6"
Jul 23 23:40:51.860131 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:51.860100    2578 scope.go:117] "RemoveContainer" containerID="0e00b447955304f7ca6ae367a974337b5d9be9694945805587a6e29a0cd277b3"
Jul 23 23:40:51.860387 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:40:51.860364    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e00b447955304f7ca6ae367a974337b5d9be9694945805587a6e29a0cd277b3\": container with ID starting with 0e00b447955304f7ca6ae367a974337b5d9be9694945805587a6e29a0cd277b3 not found: ID does not exist" containerID="0e00b447955304f7ca6ae367a974337b5d9be9694945805587a6e29a0cd277b3"
Jul 23 23:40:51.860456 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:51.860395    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e00b447955304f7ca6ae367a974337b5d9be9694945805587a6e29a0cd277b3"} err="failed to get container status \"0e00b447955304f7ca6ae367a974337b5d9be9694945805587a6e29a0cd277b3\": rpc error: code = NotFound desc = could not find container \"0e00b447955304f7ca6ae367a974337b5d9be9694945805587a6e29a0cd277b3\": container with ID starting with 0e00b447955304f7ca6ae367a974337b5d9be9694945805587a6e29a0cd277b3 not found: ID does not exist"
Jul 23 23:40:51.860456 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:51.860413    2578 scope.go:117] "RemoveContainer" containerID="92f1d78500b39abe9da1fdb426b8720163949c88975f0ebfa4e3814001168db6"
Jul 23 23:40:51.860887 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:40:51.860862    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92f1d78500b39abe9da1fdb426b8720163949c88975f0ebfa4e3814001168db6\": container with ID starting with 92f1d78500b39abe9da1fdb426b8720163949c88975f0ebfa4e3814001168db6 not found: ID does not exist" containerID="92f1d78500b39abe9da1fdb426b8720163949c88975f0ebfa4e3814001168db6"
Jul 23 23:40:51.861057 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:51.860898    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92f1d78500b39abe9da1fdb426b8720163949c88975f0ebfa4e3814001168db6"} err="failed to get container status \"92f1d78500b39abe9da1fdb426b8720163949c88975f0ebfa4e3814001168db6\": rpc error: code = NotFound desc = could not find container \"92f1d78500b39abe9da1fdb426b8720163949c88975f0ebfa4e3814001168db6\": container with ID starting with 92f1d78500b39abe9da1fdb426b8720163949c88975f0ebfa4e3814001168db6 not found: ID does not exist"
Jul 23 23:40:51.863781 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:51.863760    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm"]
Jul 23 23:40:51.867234 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:51.867213    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-17320d85-openshift-default-789457c6dd-jfcgm"]
Jul 23 23:40:53.236047 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:53.236016    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="900d43b9-392b-4782-8e17-811c52632bdf" path="/var/lib/kubelet/pods/900d43b9-392b-4782-8e17-811c52632bdf/volumes"
Jul 23 23:40:57.850139 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:40:57.850105    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7cb9a6a8-8584969b75-vs5b5"
Jul 23 23:41:02.900698 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:02.900577    2578 generic.go:356] "Generic (PLEG): container finished" podID="ac020e93-9fb2-4630-908b-d05fbda10ec0" containerID="7ade4f3356d6be4f4e1bd40e6560717fc2f0310670d66adc7f1976047492c4f7" exitCode=0
Jul 23 23:41:02.900698 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:02.900663    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq" event={"ID":"ac020e93-9fb2-4630-908b-d05fbda10ec0","Type":"ContainerDied","Data":"7ade4f3356d6be4f4e1bd40e6560717fc2f0310670d66adc7f1976047492c4f7"}
Jul 23 23:41:02.901133 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:02.901119    2578 scope.go:117] "RemoveContainer" containerID="7ade4f3356d6be4f4e1bd40e6560717fc2f0310670d66adc7f1976047492c4f7"
Jul 23 23:41:02.902158 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:02.902132    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:41:02.902232 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:02.902215    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:41:02.902287 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:02.902273    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:41:03.822480 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:03.822434    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"
Jul 23 23:41:03.908339 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:03.907376    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq" event={"ID":"ac020e93-9fb2-4630-908b-d05fbda10ec0","Type":"ContainerStarted","Data":"2ee60cc6e5a7a3c2a31612a41713001d5c63c1c978a560e5fd18c20d69001d41"}
Jul 23 23:41:04.252322 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:04.252284    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"
Jul 23 23:41:04.253542 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:04.253513    2578 patch_prober.go:28] interesting pod/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Jul 23 23:41:04.253689 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:04.253564    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq" podUID="ac020e93-9fb2-4630-908b-d05fbda10ec0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Jul 23 23:41:05.252328 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:05.252298    2578 patch_prober.go:28] interesting pod/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Jul 23 23:41:05.252744 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:05.252347    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq" podUID="ac020e93-9fb2-4630-908b-d05fbda10ec0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Jul 23 23:41:06.252622 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:06.252581    2578 patch_prober.go:28] interesting pod/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Jul 23 23:41:06.253054 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:06.252698    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq" podUID="ac020e93-9fb2-4630-908b-d05fbda10ec0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Jul 23 23:41:07.252544 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:07.252502    2578 patch_prober.go:28] interesting pod/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Jul 23 23:41:07.252742 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:07.252567    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq" podUID="ac020e93-9fb2-4630-908b-d05fbda10ec0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Jul 23 23:41:08.256767 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:08.256735    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"
Jul 23 23:41:08.257232 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:08.257090    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"
Jul 23 23:41:08.257800 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:08.257779    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"
Jul 23 23:41:24.689417 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:24.689377    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7cb9a6a8-8584969b75-vs5b5"]
Jul 23 23:41:24.689952 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:24.689636    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7cb9a6a8-8584969b75-vs5b5" podUID="ac64aeee-7def-41e3-ac9c-d26daed92052" containerName="maas-api" containerID="cri-o://a4c1e34a54510c331a7d263b6c517ab8cbc7e929bf621178fca3dee46f6c577f" gracePeriod=30
Jul 23 23:41:24.943459 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:24.943396    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7cb9a6a8-8584969b75-vs5b5"
Jul 23 23:41:25.006819 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:25.006776    2578 generic.go:356] "Generic (PLEG): container finished" podID="ac64aeee-7def-41e3-ac9c-d26daed92052" containerID="a4c1e34a54510c331a7d263b6c517ab8cbc7e929bf621178fca3dee46f6c577f" exitCode=0
Jul 23 23:41:25.007010 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:25.006833    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7cb9a6a8-8584969b75-vs5b5"
Jul 23 23:41:25.007010 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:25.006873    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7cb9a6a8-8584969b75-vs5b5" event={"ID":"ac64aeee-7def-41e3-ac9c-d26daed92052","Type":"ContainerDied","Data":"a4c1e34a54510c331a7d263b6c517ab8cbc7e929bf621178fca3dee46f6c577f"}
Jul 23 23:41:25.007010 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:25.006927    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7cb9a6a8-8584969b75-vs5b5" event={"ID":"ac64aeee-7def-41e3-ac9c-d26daed92052","Type":"ContainerDied","Data":"317e66623c68754842ae9560d99db376d35d4ddcd184b86c51f15ee003fe133a"}
Jul 23 23:41:25.007010 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:25.006952    2578 scope.go:117] "RemoveContainer" containerID="a4c1e34a54510c331a7d263b6c517ab8cbc7e929bf621178fca3dee46f6c577f"
Jul 23 23:41:25.015740 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:25.015722    2578 scope.go:117] "RemoveContainer" containerID="a4c1e34a54510c331a7d263b6c517ab8cbc7e929bf621178fca3dee46f6c577f"
Jul 23 23:41:25.016025 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:41:25.016006    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4c1e34a54510c331a7d263b6c517ab8cbc7e929bf621178fca3dee46f6c577f\": container with ID starting with a4c1e34a54510c331a7d263b6c517ab8cbc7e929bf621178fca3dee46f6c577f not found: ID does not exist" containerID="a4c1e34a54510c331a7d263b6c517ab8cbc7e929bf621178fca3dee46f6c577f"
Jul 23 23:41:25.016112 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:25.016041    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4c1e34a54510c331a7d263b6c517ab8cbc7e929bf621178fca3dee46f6c577f"} err="failed to get container status \"a4c1e34a54510c331a7d263b6c517ab8cbc7e929bf621178fca3dee46f6c577f\": rpc error: code = NotFound desc = could not find container \"a4c1e34a54510c331a7d263b6c517ab8cbc7e929bf621178fca3dee46f6c577f\": container with ID starting with a4c1e34a54510c331a7d263b6c517ab8cbc7e929bf621178fca3dee46f6c577f not found: ID does not exist"
Jul 23 23:41:25.047852 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:25.047822    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kr57b\" (UniqueName: \"kubernetes.io/projected/ac64aeee-7def-41e3-ac9c-d26daed92052-kube-api-access-kr57b\") pod \"ac64aeee-7def-41e3-ac9c-d26daed92052\" (UID: \"ac64aeee-7def-41e3-ac9c-d26daed92052\") "
Jul 23 23:41:25.047960 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:25.047942    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac64aeee-7def-41e3-ac9c-d26daed92052-openshift-service-ca\") pod \"ac64aeee-7def-41e3-ac9c-d26daed92052\" (UID: \"ac64aeee-7def-41e3-ac9c-d26daed92052\") "
Jul 23 23:41:25.048053 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:25.048040    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac64aeee-7def-41e3-ac9c-d26daed92052-maas-api-tls\") pod \"ac64aeee-7def-41e3-ac9c-d26daed92052\" (UID: \"ac64aeee-7def-41e3-ac9c-d26daed92052\") "
Jul 23 23:41:25.048301 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:25.048279    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac64aeee-7def-41e3-ac9c-d26daed92052-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ac64aeee-7def-41e3-ac9c-d26daed92052" (UID: "ac64aeee-7def-41e3-ac9c-d26daed92052"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:41:25.050003 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:25.049975    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac64aeee-7def-41e3-ac9c-d26daed92052-kube-api-access-kr57b" (OuterVolumeSpecName: "kube-api-access-kr57b") pod "ac64aeee-7def-41e3-ac9c-d26daed92052" (UID: "ac64aeee-7def-41e3-ac9c-d26daed92052"). InnerVolumeSpecName "kube-api-access-kr57b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:41:25.050083 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:25.050047    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac64aeee-7def-41e3-ac9c-d26daed92052-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ac64aeee-7def-41e3-ac9c-d26daed92052" (UID: "ac64aeee-7def-41e3-ac9c-d26daed92052"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 23:41:25.149826 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:25.149775    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac64aeee-7def-41e3-ac9c-d26daed92052-openshift-service-ca\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:41:25.149826 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:25.149819    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac64aeee-7def-41e3-ac9c-d26daed92052-maas-api-tls\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:41:25.149826 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:25.149829    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kr57b\" (UniqueName: \"kubernetes.io/projected/ac64aeee-7def-41e3-ac9c-d26daed92052-kube-api-access-kr57b\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:41:25.323431 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:25.323395    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7cb9a6a8-8584969b75-vs5b5"]
Jul 23 23:41:25.332417 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:25.327616    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7cb9a6a8-8584969b75-vs5b5"]
Jul 23 23:41:27.236066 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:27.236030    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac64aeee-7def-41e3-ac9c-d26daed92052" path="/var/lib/kubelet/pods/ac64aeee-7def-41e3-ac9c-d26daed92052/volumes"
Jul 23 23:41:29.336647 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:29.336613    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"]
Jul 23 23:41:29.337125 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:29.336904    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq" podUID="ac020e93-9fb2-4630-908b-d05fbda10ec0" containerName="istio-proxy" containerID="cri-o://2ee60cc6e5a7a3c2a31612a41713001d5c63c1c978a560e5fd18c20d69001d41" gracePeriod=30
Jul 23 23:41:34.583526 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:34.583492    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"
Jul 23 23:41:34.632111 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:34.632022    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac020e93-9fb2-4630-908b-d05fbda10ec0-istio-envoy\") pod \"ac020e93-9fb2-4630-908b-d05fbda10ec0\" (UID: \"ac020e93-9fb2-4630-908b-d05fbda10ec0\") "
Jul 23 23:41:34.632111 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:34.632080    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac020e93-9fb2-4630-908b-d05fbda10ec0-istio-podinfo\") pod \"ac020e93-9fb2-4630-908b-d05fbda10ec0\" (UID: \"ac020e93-9fb2-4630-908b-d05fbda10ec0\") "
Jul 23 23:41:34.632111 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:34.632105    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac020e93-9fb2-4630-908b-d05fbda10ec0-workload-socket\") pod \"ac020e93-9fb2-4630-908b-d05fbda10ec0\" (UID: \"ac020e93-9fb2-4630-908b-d05fbda10ec0\") "
Jul 23 23:41:34.632383 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:34.632141    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac020e93-9fb2-4630-908b-d05fbda10ec0-istio-token\") pod \"ac020e93-9fb2-4630-908b-d05fbda10ec0\" (UID: \"ac020e93-9fb2-4630-908b-d05fbda10ec0\") "
Jul 23 23:41:34.632383 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:34.632219    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac020e93-9fb2-4630-908b-d05fbda10ec0-istiod-ca-cert\") pod \"ac020e93-9fb2-4630-908b-d05fbda10ec0\" (UID: \"ac020e93-9fb2-4630-908b-d05fbda10ec0\") "
Jul 23 23:41:34.632383 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:34.632297    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac020e93-9fb2-4630-908b-d05fbda10ec0-workload-certs\") pod \"ac020e93-9fb2-4630-908b-d05fbda10ec0\" (UID: \"ac020e93-9fb2-4630-908b-d05fbda10ec0\") "
Jul 23 23:41:34.632383 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:34.632345    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac020e93-9fb2-4630-908b-d05fbda10ec0-istio-data\") pod \"ac020e93-9fb2-4630-908b-d05fbda10ec0\" (UID: \"ac020e93-9fb2-4630-908b-d05fbda10ec0\") "
Jul 23 23:41:34.632383 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:34.632380    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8ntl5\" (UniqueName: \"kubernetes.io/projected/ac020e93-9fb2-4630-908b-d05fbda10ec0-kube-api-access-8ntl5\") pod \"ac020e93-9fb2-4630-908b-d05fbda10ec0\" (UID: \"ac020e93-9fb2-4630-908b-d05fbda10ec0\") "
Jul 23 23:41:34.632640 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:34.632420    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac020e93-9fb2-4630-908b-d05fbda10ec0-credential-socket\") pod \"ac020e93-9fb2-4630-908b-d05fbda10ec0\" (UID: \"ac020e93-9fb2-4630-908b-d05fbda10ec0\") "
Jul 23 23:41:34.632640 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:34.632426    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac020e93-9fb2-4630-908b-d05fbda10ec0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ac020e93-9fb2-4630-908b-d05fbda10ec0" (UID: "ac020e93-9fb2-4630-908b-d05fbda10ec0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:41:34.632793 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:34.632620    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac020e93-9fb2-4630-908b-d05fbda10ec0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ac020e93-9fb2-4630-908b-d05fbda10ec0" (UID: "ac020e93-9fb2-4630-908b-d05fbda10ec0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:41:34.632853 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:34.632798    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac020e93-9fb2-4630-908b-d05fbda10ec0-workload-certs\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:41:34.632853 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:34.632820    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac020e93-9fb2-4630-908b-d05fbda10ec0-workload-socket\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:41:34.632945 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:34.632865    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac020e93-9fb2-4630-908b-d05fbda10ec0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ac020e93-9fb2-4630-908b-d05fbda10ec0" (UID: "ac020e93-9fb2-4630-908b-d05fbda10ec0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:41:34.632945 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:34.632891    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac020e93-9fb2-4630-908b-d05fbda10ec0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ac020e93-9fb2-4630-908b-d05fbda10ec0" (UID: "ac020e93-9fb2-4630-908b-d05fbda10ec0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:41:34.633796 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:34.633765    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac020e93-9fb2-4630-908b-d05fbda10ec0-istio-data" (OuterVolumeSpecName: "istio-data") pod "ac020e93-9fb2-4630-908b-d05fbda10ec0" (UID: "ac020e93-9fb2-4630-908b-d05fbda10ec0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:41:34.635041 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:34.635010    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac020e93-9fb2-4630-908b-d05fbda10ec0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ac020e93-9fb2-4630-908b-d05fbda10ec0" (UID: "ac020e93-9fb2-4630-908b-d05fbda10ec0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:41:34.635148 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:34.635012    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ac020e93-9fb2-4630-908b-d05fbda10ec0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ac020e93-9fb2-4630-908b-d05fbda10ec0" (UID: "ac020e93-9fb2-4630-908b-d05fbda10ec0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 23:41:34.635148 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:34.635061    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac020e93-9fb2-4630-908b-d05fbda10ec0-istio-token" (OuterVolumeSpecName: "istio-token") pod "ac020e93-9fb2-4630-908b-d05fbda10ec0" (UID: "ac020e93-9fb2-4630-908b-d05fbda10ec0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:41:34.635148 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:34.635077    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac020e93-9fb2-4630-908b-d05fbda10ec0-kube-api-access-8ntl5" (OuterVolumeSpecName: "kube-api-access-8ntl5") pod "ac020e93-9fb2-4630-908b-d05fbda10ec0" (UID: "ac020e93-9fb2-4630-908b-d05fbda10ec0"). InnerVolumeSpecName "kube-api-access-8ntl5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:41:34.733443 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:34.733399    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac020e93-9fb2-4630-908b-d05fbda10ec0-istio-data\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:41:34.733443 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:34.733434    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ntl5\" (UniqueName: \"kubernetes.io/projected/ac020e93-9fb2-4630-908b-d05fbda10ec0-kube-api-access-8ntl5\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:41:34.733443 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:34.733449    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac020e93-9fb2-4630-908b-d05fbda10ec0-credential-socket\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:41:34.733728 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:34.733461    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac020e93-9fb2-4630-908b-d05fbda10ec0-istio-envoy\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:41:34.733728 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:34.733474    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac020e93-9fb2-4630-908b-d05fbda10ec0-istio-podinfo\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:41:34.733728 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:34.733485    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac020e93-9fb2-4630-908b-d05fbda10ec0-istio-token\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:41:34.733728 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:34.733496    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac020e93-9fb2-4630-908b-d05fbda10ec0-istiod-ca-cert\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:41:35.064932 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:35.064889    2578 generic.go:356] "Generic (PLEG): container finished" podID="ac020e93-9fb2-4630-908b-d05fbda10ec0" containerID="2ee60cc6e5a7a3c2a31612a41713001d5c63c1c978a560e5fd18c20d69001d41" exitCode=0
Jul 23 23:41:35.065101 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:35.064939    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"
Jul 23 23:41:35.065101 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:35.064981    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq" event={"ID":"ac020e93-9fb2-4630-908b-d05fbda10ec0","Type":"ContainerDied","Data":"2ee60cc6e5a7a3c2a31612a41713001d5c63c1c978a560e5fd18c20d69001d41"}
Jul 23 23:41:35.065101 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:35.065042    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq" event={"ID":"ac020e93-9fb2-4630-908b-d05fbda10ec0","Type":"ContainerDied","Data":"539b30985f080a876853f288ff5c66419d1024ff86a33a918ee845ff674a432a"}
Jul 23 23:41:35.065101 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:35.065067    2578 scope.go:117] "RemoveContainer" containerID="2ee60cc6e5a7a3c2a31612a41713001d5c63c1c978a560e5fd18c20d69001d41"
Jul 23 23:41:35.074101 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:35.074085    2578 scope.go:117] "RemoveContainer" containerID="7ade4f3356d6be4f4e1bd40e6560717fc2f0310670d66adc7f1976047492c4f7"
Jul 23 23:41:35.081899 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:35.081883    2578 scope.go:117] "RemoveContainer" containerID="2ee60cc6e5a7a3c2a31612a41713001d5c63c1c978a560e5fd18c20d69001d41"
Jul 23 23:41:35.082161 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:41:35.082141    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ee60cc6e5a7a3c2a31612a41713001d5c63c1c978a560e5fd18c20d69001d41\": container with ID starting with 2ee60cc6e5a7a3c2a31612a41713001d5c63c1c978a560e5fd18c20d69001d41 not found: ID does not exist" containerID="2ee60cc6e5a7a3c2a31612a41713001d5c63c1c978a560e5fd18c20d69001d41"
Jul 23 23:41:35.082207 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:35.082172    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ee60cc6e5a7a3c2a31612a41713001d5c63c1c978a560e5fd18c20d69001d41"} err="failed to get container status \"2ee60cc6e5a7a3c2a31612a41713001d5c63c1c978a560e5fd18c20d69001d41\": rpc error: code = NotFound desc = could not find container \"2ee60cc6e5a7a3c2a31612a41713001d5c63c1c978a560e5fd18c20d69001d41\": container with ID starting with 2ee60cc6e5a7a3c2a31612a41713001d5c63c1c978a560e5fd18c20d69001d41 not found: ID does not exist"
Jul 23 23:41:35.082207 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:35.082190    2578 scope.go:117] "RemoveContainer" containerID="7ade4f3356d6be4f4e1bd40e6560717fc2f0310670d66adc7f1976047492c4f7"
Jul 23 23:41:35.082427 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:41:35.082411    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ade4f3356d6be4f4e1bd40e6560717fc2f0310670d66adc7f1976047492c4f7\": container with ID starting with 7ade4f3356d6be4f4e1bd40e6560717fc2f0310670d66adc7f1976047492c4f7 not found: ID does not exist" containerID="7ade4f3356d6be4f4e1bd40e6560717fc2f0310670d66adc7f1976047492c4f7"
Jul 23 23:41:35.082463 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:35.082434    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ade4f3356d6be4f4e1bd40e6560717fc2f0310670d66adc7f1976047492c4f7"} err="failed to get container status \"7ade4f3356d6be4f4e1bd40e6560717fc2f0310670d66adc7f1976047492c4f7\": rpc error: code = NotFound desc = could not find container \"7ade4f3356d6be4f4e1bd40e6560717fc2f0310670d66adc7f1976047492c4f7\": container with ID starting with 7ade4f3356d6be4f4e1bd40e6560717fc2f0310670d66adc7f1976047492c4f7 not found: ID does not exist"
Jul 23 23:41:35.087182 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:35.087161    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"]
Jul 23 23:41:35.090450 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:35.090426    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-7cb9a6a8-openshift-default-5c59b5ccf6-8f4rq"]
Jul 23 23:41:35.235841 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:41:35.235806    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac020e93-9fb2-4630-908b-d05fbda10ec0" path="/var/lib/kubelet/pods/ac020e93-9fb2-4630-908b-d05fbda10ec0/volumes"
Jul 23 23:43:06.291709 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.291668    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5"]
Jul 23 23:43:06.292334 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.292012    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac020e93-9fb2-4630-908b-d05fbda10ec0" containerName="istio-proxy"
Jul 23 23:43:06.292334 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.292023    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac020e93-9fb2-4630-908b-d05fbda10ec0" containerName="istio-proxy"
Jul 23 23:43:06.292334 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.292031    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac020e93-9fb2-4630-908b-d05fbda10ec0" containerName="istio-proxy"
Jul 23 23:43:06.292334 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.292037    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac020e93-9fb2-4630-908b-d05fbda10ec0" containerName="istio-proxy"
Jul 23 23:43:06.292334 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.292045    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac64aeee-7def-41e3-ac9c-d26daed92052" containerName="maas-api"
Jul 23 23:43:06.292334 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.292050    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac64aeee-7def-41e3-ac9c-d26daed92052" containerName="maas-api"
Jul 23 23:43:06.292334 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.292062    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="900d43b9-392b-4782-8e17-811c52632bdf" containerName="istio-proxy"
Jul 23 23:43:06.292334 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.292068    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="900d43b9-392b-4782-8e17-811c52632bdf" containerName="istio-proxy"
Jul 23 23:43:06.292334 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.292087    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="900d43b9-392b-4782-8e17-811c52632bdf" containerName="istio-proxy"
Jul 23 23:43:06.292334 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.292092    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="900d43b9-392b-4782-8e17-811c52632bdf" containerName="istio-proxy"
Jul 23 23:43:06.292334 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.292147    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="900d43b9-392b-4782-8e17-811c52632bdf" containerName="istio-proxy"
Jul 23 23:43:06.292334 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.292157    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac64aeee-7def-41e3-ac9c-d26daed92052" containerName="maas-api"
Jul 23 23:43:06.292334 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.292164    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac020e93-9fb2-4630-908b-d05fbda10ec0" containerName="istio-proxy"
Jul 23 23:43:06.292334 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.292170    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac020e93-9fb2-4630-908b-d05fbda10ec0" containerName="istio-proxy"
Jul 23 23:43:06.294629 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.294615    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5"
Jul 23 23:43:06.297675 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.297639    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-4155fe38-openshift-default-dockercfg-cvjgg\""
Jul 23 23:43:06.314914 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.314884    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5"]
Jul 23 23:43:06.410349 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.410314    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e483809-56dc-44d5-8f82-e4ed6daf6815-istio-data\") pod \"e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5\" (UID: \"4e483809-56dc-44d5-8f82-e4ed6daf6815\") " pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5"
Jul 23 23:43:06.410349 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.410349    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e483809-56dc-44d5-8f82-e4ed6daf6815-istio-token\") pod \"e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5\" (UID: \"4e483809-56dc-44d5-8f82-e4ed6daf6815\") " pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5"
Jul 23 23:43:06.410601 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.410449    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e483809-56dc-44d5-8f82-e4ed6daf6815-workload-socket\") pod \"e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5\" (UID: \"4e483809-56dc-44d5-8f82-e4ed6daf6815\") " pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5"
Jul 23 23:43:06.410601 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.410509    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e483809-56dc-44d5-8f82-e4ed6daf6815-istio-envoy\") pod \"e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5\" (UID: \"4e483809-56dc-44d5-8f82-e4ed6daf6815\") " pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5"
Jul 23 23:43:06.410601 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.410569    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e483809-56dc-44d5-8f82-e4ed6daf6815-istio-podinfo\") pod \"e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5\" (UID: \"4e483809-56dc-44d5-8f82-e4ed6daf6815\") " pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5"
Jul 23 23:43:06.410758 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.410612    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e483809-56dc-44d5-8f82-e4ed6daf6815-workload-certs\") pod \"e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5\" (UID: \"4e483809-56dc-44d5-8f82-e4ed6daf6815\") " pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5"
Jul 23 23:43:06.410758 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.410680    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8p4kt\" (UniqueName: \"kubernetes.io/projected/4e483809-56dc-44d5-8f82-e4ed6daf6815-kube-api-access-8p4kt\") pod \"e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5\" (UID: \"4e483809-56dc-44d5-8f82-e4ed6daf6815\") " pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5"
Jul 23 23:43:06.410758 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.410721    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e483809-56dc-44d5-8f82-e4ed6daf6815-istiod-ca-cert\") pod \"e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5\" (UID: \"4e483809-56dc-44d5-8f82-e4ed6daf6815\") " pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5"
Jul 23 23:43:06.410758 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.410739    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e483809-56dc-44d5-8f82-e4ed6daf6815-credential-socket\") pod \"e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5\" (UID: \"4e483809-56dc-44d5-8f82-e4ed6daf6815\") " pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5"
Jul 23 23:43:06.512093 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.512056    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8p4kt\" (UniqueName: \"kubernetes.io/projected/4e483809-56dc-44d5-8f82-e4ed6daf6815-kube-api-access-8p4kt\") pod \"e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5\" (UID: \"4e483809-56dc-44d5-8f82-e4ed6daf6815\") " pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5"
Jul 23 23:43:06.512093 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.512096    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e483809-56dc-44d5-8f82-e4ed6daf6815-istiod-ca-cert\") pod \"e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5\" (UID: \"4e483809-56dc-44d5-8f82-e4ed6daf6815\") " pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5"
Jul 23 23:43:06.512343 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.512117    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e483809-56dc-44d5-8f82-e4ed6daf6815-credential-socket\") pod \"e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5\" (UID: \"4e483809-56dc-44d5-8f82-e4ed6daf6815\") " pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5"
Jul 23 23:43:06.512343 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.512150    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e483809-56dc-44d5-8f82-e4ed6daf6815-istio-data\") pod \"e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5\" (UID: \"4e483809-56dc-44d5-8f82-e4ed6daf6815\") " pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5"
Jul 23 23:43:06.512343 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.512175    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e483809-56dc-44d5-8f82-e4ed6daf6815-istio-token\") pod \"e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5\" (UID: \"4e483809-56dc-44d5-8f82-e4ed6daf6815\") " pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5"
Jul 23 23:43:06.512343 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.512247    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e483809-56dc-44d5-8f82-e4ed6daf6815-workload-socket\") pod \"e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5\" (UID: \"4e483809-56dc-44d5-8f82-e4ed6daf6815\") " pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5"
Jul 23 23:43:06.512343 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.512294    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e483809-56dc-44d5-8f82-e4ed6daf6815-istio-envoy\") pod \"e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5\" (UID: \"4e483809-56dc-44d5-8f82-e4ed6daf6815\") " pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5"
Jul 23 23:43:06.512343 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.512319    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e483809-56dc-44d5-8f82-e4ed6daf6815-istio-podinfo\") pod \"e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5\" (UID: \"4e483809-56dc-44d5-8f82-e4ed6daf6815\") " pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5"
Jul 23 23:43:06.512700 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.512357    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e483809-56dc-44d5-8f82-e4ed6daf6815-workload-certs\") pod \"e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5\" (UID: \"4e483809-56dc-44d5-8f82-e4ed6daf6815\") " pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5"
Jul 23 23:43:06.512700 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.512559    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e483809-56dc-44d5-8f82-e4ed6daf6815-credential-socket\") pod \"e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5\" (UID: \"4e483809-56dc-44d5-8f82-e4ed6daf6815\") " pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5"
Jul 23 23:43:06.512700 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.512613    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e483809-56dc-44d5-8f82-e4ed6daf6815-istio-data\") pod \"e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5\" (UID: \"4e483809-56dc-44d5-8f82-e4ed6daf6815\") " pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5"
Jul 23 23:43:06.512830 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.512768    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e483809-56dc-44d5-8f82-e4ed6daf6815-workload-socket\") pod \"e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5\" (UID: \"4e483809-56dc-44d5-8f82-e4ed6daf6815\") " pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5"
Jul 23 23:43:06.512877 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.512853    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e483809-56dc-44d5-8f82-e4ed6daf6815-workload-certs\") pod \"e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5\" (UID: \"4e483809-56dc-44d5-8f82-e4ed6daf6815\") " pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5"
Jul 23 23:43:06.513019 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.512993    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e483809-56dc-44d5-8f82-e4ed6daf6815-istiod-ca-cert\") pod \"e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5\" (UID: \"4e483809-56dc-44d5-8f82-e4ed6daf6815\") " pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5"
Jul 23 23:43:06.514816 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.514791    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e483809-56dc-44d5-8f82-e4ed6daf6815-istio-envoy\") pod \"e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5\" (UID: \"4e483809-56dc-44d5-8f82-e4ed6daf6815\") " pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5"
Jul 23 23:43:06.515116 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.515093    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e483809-56dc-44d5-8f82-e4ed6daf6815-istio-podinfo\") pod \"e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5\" (UID: \"4e483809-56dc-44d5-8f82-e4ed6daf6815\") " pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5"
Jul 23 23:43:06.519968 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.519947    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e483809-56dc-44d5-8f82-e4ed6daf6815-istio-token\") pod \"e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5\" (UID: \"4e483809-56dc-44d5-8f82-e4ed6daf6815\") " pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5"
Jul 23 23:43:06.520765 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.520743    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8p4kt\" (UniqueName: \"kubernetes.io/projected/4e483809-56dc-44d5-8f82-e4ed6daf6815-kube-api-access-8p4kt\") pod \"e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5\" (UID: \"4e483809-56dc-44d5-8f82-e4ed6daf6815\") " pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5"
Jul 23 23:43:06.605120 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.605032    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5"
Jul 23 23:43:06.744324 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.744296    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5"]
Jul 23 23:43:06.746381 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:43:06.746356    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e483809_56dc_44d5_8f82_e4ed6daf6815.slice/crio-b03d9e9f2f72687f1e080dc2ab78c0c3737d04636fb692f6a391308ead996b6f WatchSource:0}: Error finding container b03d9e9f2f72687f1e080dc2ab78c0c3737d04636fb692f6a391308ead996b6f: Status 404 returned error can't find the container with id b03d9e9f2f72687f1e080dc2ab78c0c3737d04636fb692f6a391308ead996b6f
Jul 23 23:43:06.748856 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.748823    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:43:06.748944 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.748888    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:43:06.748944 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:06.748915    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:43:07.497058 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:07.497018    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5" event={"ID":"4e483809-56dc-44d5-8f82-e4ed6daf6815","Type":"ContainerStarted","Data":"b40f072f7b56595475ecbc79fe2f597ce36e5b402ba4ecd891fadd36d4d6f46e"}
Jul 23 23:43:07.497443 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:07.497082    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5" event={"ID":"4e483809-56dc-44d5-8f82-e4ed6daf6815","Type":"ContainerStarted","Data":"b03d9e9f2f72687f1e080dc2ab78c0c3737d04636fb692f6a391308ead996b6f"}
Jul 23 23:43:07.520907 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:07.520850    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5" podStartSLOduration=1.5208356859999999 podStartE2EDuration="1.520835686s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:43:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:43:07.518125926 +0000 UTC m=+1776.887651586" watchObservedRunningTime="2026-07-23 23:43:07.520835686 +0000 UTC m=+1776.890361409"
Jul 23 23:43:07.605724 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:07.605693    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5"
Jul 23 23:43:07.610587 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:07.610564    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5"
Jul 23 23:43:08.502417 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:08.502384    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5"
Jul 23 23:43:08.503305 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:08.503280    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5"
Jul 23 23:43:09.611135 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:09.611104    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4155fe38-6b7447955-szdv5"]
Jul 23 23:43:09.611590 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:09.611521    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="900d43b9-392b-4782-8e17-811c52632bdf" containerName="istio-proxy"
Jul 23 23:43:09.614535 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:09.614514    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4155fe38-6b7447955-szdv5"
Jul 23 23:43:09.617493 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:09.617470    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-4155fe38-serving-cert\""
Jul 23 23:43:09.625841 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:09.625817    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4155fe38-6b7447955-szdv5"]
Jul 23 23:43:09.742836 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:09.742797    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f7565a8-5332-4653-8ee7-8d0031962fb4-openshift-service-ca\") pod \"maas-api-e2e-mt-4155fe38-6b7447955-szdv5\" (UID: \"9f7565a8-5332-4653-8ee7-8d0031962fb4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4155fe38-6b7447955-szdv5"
Jul 23 23:43:09.742836 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:09.742838    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2djq\" (UniqueName: \"kubernetes.io/projected/9f7565a8-5332-4653-8ee7-8d0031962fb4-kube-api-access-f2djq\") pod \"maas-api-e2e-mt-4155fe38-6b7447955-szdv5\" (UID: \"9f7565a8-5332-4653-8ee7-8d0031962fb4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4155fe38-6b7447955-szdv5"
Jul 23 23:43:09.743085 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:09.742886    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f7565a8-5332-4653-8ee7-8d0031962fb4-maas-api-tls\") pod \"maas-api-e2e-mt-4155fe38-6b7447955-szdv5\" (UID: \"9f7565a8-5332-4653-8ee7-8d0031962fb4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4155fe38-6b7447955-szdv5"
Jul 23 23:43:09.844080 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:09.844042    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f7565a8-5332-4653-8ee7-8d0031962fb4-maas-api-tls\") pod \"maas-api-e2e-mt-4155fe38-6b7447955-szdv5\" (UID: \"9f7565a8-5332-4653-8ee7-8d0031962fb4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4155fe38-6b7447955-szdv5"
Jul 23 23:43:09.844255 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:09.844141    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f7565a8-5332-4653-8ee7-8d0031962fb4-openshift-service-ca\") pod \"maas-api-e2e-mt-4155fe38-6b7447955-szdv5\" (UID: \"9f7565a8-5332-4653-8ee7-8d0031962fb4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4155fe38-6b7447955-szdv5"
Jul 23 23:43:09.844255 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:09.844164    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2djq\" (UniqueName: \"kubernetes.io/projected/9f7565a8-5332-4653-8ee7-8d0031962fb4-kube-api-access-f2djq\") pod \"maas-api-e2e-mt-4155fe38-6b7447955-szdv5\" (UID: \"9f7565a8-5332-4653-8ee7-8d0031962fb4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4155fe38-6b7447955-szdv5"
Jul 23 23:43:09.844850 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:09.844785    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f7565a8-5332-4653-8ee7-8d0031962fb4-openshift-service-ca\") pod \"maas-api-e2e-mt-4155fe38-6b7447955-szdv5\" (UID: \"9f7565a8-5332-4653-8ee7-8d0031962fb4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4155fe38-6b7447955-szdv5"
Jul 23 23:43:09.846642 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:09.846612    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f7565a8-5332-4653-8ee7-8d0031962fb4-maas-api-tls\") pod \"maas-api-e2e-mt-4155fe38-6b7447955-szdv5\" (UID: \"9f7565a8-5332-4653-8ee7-8d0031962fb4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4155fe38-6b7447955-szdv5"
Jul 23 23:43:09.859468 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:09.859444    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2djq\" (UniqueName: \"kubernetes.io/projected/9f7565a8-5332-4653-8ee7-8d0031962fb4-kube-api-access-f2djq\") pod \"maas-api-e2e-mt-4155fe38-6b7447955-szdv5\" (UID: \"9f7565a8-5332-4653-8ee7-8d0031962fb4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4155fe38-6b7447955-szdv5"
Jul 23 23:43:09.926922 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:09.926885    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4155fe38-6b7447955-szdv5"
Jul 23 23:43:10.067026 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:10.066999    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4155fe38-6b7447955-szdv5"]
Jul 23 23:43:10.069170 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:43:10.069142    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f7565a8_5332_4653_8ee7_8d0031962fb4.slice/crio-a31e274f7f6ec10a79bc644274de1656a791761616a1a1fad1b47d6d259f0e9f WatchSource:0}: Error finding container a31e274f7f6ec10a79bc644274de1656a791761616a1a1fad1b47d6d259f0e9f: Status 404 returned error can't find the container with id a31e274f7f6ec10a79bc644274de1656a791761616a1a1fad1b47d6d259f0e9f
Jul 23 23:43:10.513243 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:10.513218    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4155fe38-6b7447955-szdv5" event={"ID":"9f7565a8-5332-4653-8ee7-8d0031962fb4","Type":"ContainerStarted","Data":"a31e274f7f6ec10a79bc644274de1656a791761616a1a1fad1b47d6d259f0e9f"}
Jul 23 23:43:11.520299 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:11.520260    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4155fe38-6b7447955-szdv5" event={"ID":"9f7565a8-5332-4653-8ee7-8d0031962fb4","Type":"ContainerStarted","Data":"bc590d367887ab78b43583a7fbad9c6c76e511d6d61bca55474c32e7ffa5b11b"}
Jul 23 23:43:11.520842 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:11.520421    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4155fe38-6b7447955-szdv5"
Jul 23 23:43:11.539487 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:11.539434    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4155fe38-6b7447955-szdv5" podStartSLOduration=2.101628154 podStartE2EDuration="2.539413679s" totalImagesPullingTime="437.785525ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:43:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:43:11.537313809 +0000 UTC m=+1780.906839469" watchObservedRunningTime="2026-07-23 23:43:11.539413679 +0000 UTC m=+1780.908939338"
Jul 23 23:43:17.530740 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:17.530707    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4155fe38-6b7447955-szdv5"
Jul 23 23:43:21.570112 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:21.570079    2578 generic.go:356] "Generic (PLEG): container finished" podID="4e483809-56dc-44d5-8f82-e4ed6daf6815" containerID="b40f072f7b56595475ecbc79fe2f597ce36e5b402ba4ecd891fadd36d4d6f46e" exitCode=0
Jul 23 23:43:21.570542 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:21.570148    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5" event={"ID":"4e483809-56dc-44d5-8f82-e4ed6daf6815","Type":"ContainerDied","Data":"b40f072f7b56595475ecbc79fe2f597ce36e5b402ba4ecd891fadd36d4d6f46e"}
Jul 23 23:43:21.570593 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:21.570585    2578 scope.go:117] "RemoveContainer" containerID="b40f072f7b56595475ecbc79fe2f597ce36e5b402ba4ecd891fadd36d4d6f46e"
Jul 23 23:43:21.571629 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:21.571605    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:43:21.571710 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:21.571687    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:43:21.571747 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:21.571720    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:43:22.588680 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:22.587971    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5" event={"ID":"4e483809-56dc-44d5-8f82-e4ed6daf6815","Type":"ContainerStarted","Data":"8a4ed1764e2b73abc4ef88963352b1eddbc7de9182014eec3f480b3e1856c5cb"}
Jul 23 23:43:22.605480 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:22.605443    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5"
Jul 23 23:43:22.606345 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:22.606323    2578 patch_prober.go:28] interesting pod/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 23 23:43:22.606407 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:22.606369    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5" podUID="4e483809-56dc-44d5-8f82-e4ed6daf6815" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 23 23:43:23.502988 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:23.502946    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5"
Jul 23 23:43:23.606038 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:23.606004    2578 patch_prober.go:28] interesting pod/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 23 23:43:23.606410 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:23.606064    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5" podUID="4e483809-56dc-44d5-8f82-e4ed6daf6815" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 23 23:43:24.605724 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:24.605690    2578 patch_prober.go:28] interesting pod/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 23 23:43:24.605872 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:24.605746    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5" podUID="4e483809-56dc-44d5-8f82-e4ed6daf6815" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 23 23:43:25.605922 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:25.605889    2578 patch_prober.go:28] interesting pod/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 23 23:43:25.606321 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:25.605952    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5" podUID="4e483809-56dc-44d5-8f82-e4ed6daf6815" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 23 23:43:26.609457 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:26.609427    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5"
Jul 23 23:43:26.610544 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:26.610523    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5"
Jul 23 23:43:31.255049 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:31.255015    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4155fe38-6b7447955-szdv5"]
Jul 23 23:43:31.255465 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:31.255288    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4155fe38-6b7447955-szdv5" podUID="9f7565a8-5332-4653-8ee7-8d0031962fb4" containerName="maas-api" containerID="cri-o://bc590d367887ab78b43583a7fbad9c6c76e511d6d61bca55474c32e7ffa5b11b" gracePeriod=30
Jul 23 23:43:31.515428 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:31.515368    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4155fe38-6b7447955-szdv5"
Jul 23 23:43:31.557624 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:31.557590    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f2djq\" (UniqueName: \"kubernetes.io/projected/9f7565a8-5332-4653-8ee7-8d0031962fb4-kube-api-access-f2djq\") pod \"9f7565a8-5332-4653-8ee7-8d0031962fb4\" (UID: \"9f7565a8-5332-4653-8ee7-8d0031962fb4\") "
Jul 23 23:43:31.557824 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:31.557682    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f7565a8-5332-4653-8ee7-8d0031962fb4-openshift-service-ca\") pod \"9f7565a8-5332-4653-8ee7-8d0031962fb4\" (UID: \"9f7565a8-5332-4653-8ee7-8d0031962fb4\") "
Jul 23 23:43:31.557824 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:31.557734    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f7565a8-5332-4653-8ee7-8d0031962fb4-maas-api-tls\") pod \"9f7565a8-5332-4653-8ee7-8d0031962fb4\" (UID: \"9f7565a8-5332-4653-8ee7-8d0031962fb4\") "
Jul 23 23:43:31.558102 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:31.558068    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f7565a8-5332-4653-8ee7-8d0031962fb4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9f7565a8-5332-4653-8ee7-8d0031962fb4" (UID: "9f7565a8-5332-4653-8ee7-8d0031962fb4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:43:31.560244 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:31.560216    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f7565a8-5332-4653-8ee7-8d0031962fb4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9f7565a8-5332-4653-8ee7-8d0031962fb4" (UID: "9f7565a8-5332-4653-8ee7-8d0031962fb4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 23:43:31.560383 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:31.560358    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f7565a8-5332-4653-8ee7-8d0031962fb4-kube-api-access-f2djq" (OuterVolumeSpecName: "kube-api-access-f2djq") pod "9f7565a8-5332-4653-8ee7-8d0031962fb4" (UID: "9f7565a8-5332-4653-8ee7-8d0031962fb4"). InnerVolumeSpecName "kube-api-access-f2djq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:43:31.631838 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:31.631797    2578 generic.go:356] "Generic (PLEG): container finished" podID="9f7565a8-5332-4653-8ee7-8d0031962fb4" containerID="bc590d367887ab78b43583a7fbad9c6c76e511d6d61bca55474c32e7ffa5b11b" exitCode=0
Jul 23 23:43:31.631838 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:31.631841    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4155fe38-6b7447955-szdv5"
Jul 23 23:43:31.632104 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:31.631852    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4155fe38-6b7447955-szdv5" event={"ID":"9f7565a8-5332-4653-8ee7-8d0031962fb4","Type":"ContainerDied","Data":"bc590d367887ab78b43583a7fbad9c6c76e511d6d61bca55474c32e7ffa5b11b"}
Jul 23 23:43:31.632104 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:31.631934    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4155fe38-6b7447955-szdv5" event={"ID":"9f7565a8-5332-4653-8ee7-8d0031962fb4","Type":"ContainerDied","Data":"a31e274f7f6ec10a79bc644274de1656a791761616a1a1fad1b47d6d259f0e9f"}
Jul 23 23:43:31.632104 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:31.631964    2578 scope.go:117] "RemoveContainer" containerID="bc590d367887ab78b43583a7fbad9c6c76e511d6d61bca55474c32e7ffa5b11b"
Jul 23 23:43:31.641222 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:31.641203    2578 scope.go:117] "RemoveContainer" containerID="bc590d367887ab78b43583a7fbad9c6c76e511d6d61bca55474c32e7ffa5b11b"
Jul 23 23:43:31.641500 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:43:31.641474    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc590d367887ab78b43583a7fbad9c6c76e511d6d61bca55474c32e7ffa5b11b\": container with ID starting with bc590d367887ab78b43583a7fbad9c6c76e511d6d61bca55474c32e7ffa5b11b not found: ID does not exist" containerID="bc590d367887ab78b43583a7fbad9c6c76e511d6d61bca55474c32e7ffa5b11b"
Jul 23 23:43:31.641573 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:31.641513    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc590d367887ab78b43583a7fbad9c6c76e511d6d61bca55474c32e7ffa5b11b"} err="failed to get container status \"bc590d367887ab78b43583a7fbad9c6c76e511d6d61bca55474c32e7ffa5b11b\": rpc error: code = NotFound desc = could not find container \"bc590d367887ab78b43583a7fbad9c6c76e511d6d61bca55474c32e7ffa5b11b\": container with ID starting with bc590d367887ab78b43583a7fbad9c6c76e511d6d61bca55474c32e7ffa5b11b not found: ID does not exist"
Jul 23 23:43:31.652917 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:31.652869    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4155fe38-6b7447955-szdv5"]
Jul 23 23:43:31.654772 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:31.654748    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4155fe38-6b7447955-szdv5"]
Jul 23 23:43:31.658760 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:31.658741    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f2djq\" (UniqueName: \"kubernetes.io/projected/9f7565a8-5332-4653-8ee7-8d0031962fb4-kube-api-access-f2djq\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:43:31.658820 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:31.658765    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f7565a8-5332-4653-8ee7-8d0031962fb4-openshift-service-ca\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:43:31.658820 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:31.658777    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f7565a8-5332-4653-8ee7-8d0031962fb4-maas-api-tls\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:43:33.236461 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:33.236425    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f7565a8-5332-4653-8ee7-8d0031962fb4" path="/var/lib/kubelet/pods/9f7565a8-5332-4653-8ee7-8d0031962fb4/volumes"
Jul 23 23:43:42.369068 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:42.369032    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5"]
Jul 23 23:43:42.369566 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:42.369250    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5" podUID="4e483809-56dc-44d5-8f82-e4ed6daf6815" containerName="istio-proxy" containerID="cri-o://8a4ed1764e2b73abc4ef88963352b1eddbc7de9182014eec3f480b3e1856c5cb" gracePeriod=30
Jul 23 23:43:47.634891 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:47.634817    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5"
Jul 23 23:43:47.704849 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:47.704812    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8p4kt\" (UniqueName: \"kubernetes.io/projected/4e483809-56dc-44d5-8f82-e4ed6daf6815-kube-api-access-8p4kt\") pod \"4e483809-56dc-44d5-8f82-e4ed6daf6815\" (UID: \"4e483809-56dc-44d5-8f82-e4ed6daf6815\") "
Jul 23 23:43:47.705051 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:47.704855    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e483809-56dc-44d5-8f82-e4ed6daf6815-istio-data\") pod \"4e483809-56dc-44d5-8f82-e4ed6daf6815\" (UID: \"4e483809-56dc-44d5-8f82-e4ed6daf6815\") "
Jul 23 23:43:47.705051 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:47.704920    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e483809-56dc-44d5-8f82-e4ed6daf6815-istio-envoy\") pod \"4e483809-56dc-44d5-8f82-e4ed6daf6815\" (UID: \"4e483809-56dc-44d5-8f82-e4ed6daf6815\") "
Jul 23 23:43:47.705051 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:47.704945    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e483809-56dc-44d5-8f82-e4ed6daf6815-istiod-ca-cert\") pod \"4e483809-56dc-44d5-8f82-e4ed6daf6815\" (UID: \"4e483809-56dc-44d5-8f82-e4ed6daf6815\") "
Jul 23 23:43:47.705051 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:47.705027    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e483809-56dc-44d5-8f82-e4ed6daf6815-credential-socket\") pod \"4e483809-56dc-44d5-8f82-e4ed6daf6815\" (UID: \"4e483809-56dc-44d5-8f82-e4ed6daf6815\") "
Jul 23 23:43:47.705333 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:47.705070    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e483809-56dc-44d5-8f82-e4ed6daf6815-istio-podinfo\") pod \"4e483809-56dc-44d5-8f82-e4ed6daf6815\" (UID: \"4e483809-56dc-44d5-8f82-e4ed6daf6815\") "
Jul 23 23:43:47.705333 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:47.705100    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e483809-56dc-44d5-8f82-e4ed6daf6815-istio-token\") pod \"4e483809-56dc-44d5-8f82-e4ed6daf6815\" (UID: \"4e483809-56dc-44d5-8f82-e4ed6daf6815\") "
Jul 23 23:43:47.705333 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:47.705139    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e483809-56dc-44d5-8f82-e4ed6daf6815-workload-socket\") pod \"4e483809-56dc-44d5-8f82-e4ed6daf6815\" (UID: \"4e483809-56dc-44d5-8f82-e4ed6daf6815\") "
Jul 23 23:43:47.705333 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:47.705196    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e483809-56dc-44d5-8f82-e4ed6daf6815-workload-certs\") pod \"4e483809-56dc-44d5-8f82-e4ed6daf6815\" (UID: \"4e483809-56dc-44d5-8f82-e4ed6daf6815\") "
Jul 23 23:43:47.705529 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:47.705343    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e483809-56dc-44d5-8f82-e4ed6daf6815-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4e483809-56dc-44d5-8f82-e4ed6daf6815" (UID: "4e483809-56dc-44d5-8f82-e4ed6daf6815"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:43:47.705583 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:47.705538    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e483809-56dc-44d5-8f82-e4ed6daf6815-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4e483809-56dc-44d5-8f82-e4ed6daf6815" (UID: "4e483809-56dc-44d5-8f82-e4ed6daf6815"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:43:47.705641 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:47.705612    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e483809-56dc-44d5-8f82-e4ed6daf6815-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4e483809-56dc-44d5-8f82-e4ed6daf6815" (UID: "4e483809-56dc-44d5-8f82-e4ed6daf6815"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:43:47.705881 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:47.705828    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e483809-56dc-44d5-8f82-e4ed6daf6815-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4e483809-56dc-44d5-8f82-e4ed6daf6815" (UID: "4e483809-56dc-44d5-8f82-e4ed6daf6815"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:43:47.705881 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:47.705842    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e483809-56dc-44d5-8f82-e4ed6daf6815-workload-certs\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:43:47.705881 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:47.705863    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e483809-56dc-44d5-8f82-e4ed6daf6815-istiod-ca-cert\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:43:47.705881 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:47.705875    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e483809-56dc-44d5-8f82-e4ed6daf6815-credential-socket\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:43:47.706128 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:47.705945    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e483809-56dc-44d5-8f82-e4ed6daf6815-istio-data" (OuterVolumeSpecName: "istio-data") pod "4e483809-56dc-44d5-8f82-e4ed6daf6815" (UID: "4e483809-56dc-44d5-8f82-e4ed6daf6815"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:43:47.707740 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:47.707713    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e483809-56dc-44d5-8f82-e4ed6daf6815-kube-api-access-8p4kt" (OuterVolumeSpecName: "kube-api-access-8p4kt") pod "4e483809-56dc-44d5-8f82-e4ed6daf6815" (UID: "4e483809-56dc-44d5-8f82-e4ed6daf6815"). InnerVolumeSpecName "kube-api-access-8p4kt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:43:47.707892 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:47.707859    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e483809-56dc-44d5-8f82-e4ed6daf6815-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4e483809-56dc-44d5-8f82-e4ed6daf6815" (UID: "4e483809-56dc-44d5-8f82-e4ed6daf6815"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:43:47.707892 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:47.707870    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4e483809-56dc-44d5-8f82-e4ed6daf6815-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4e483809-56dc-44d5-8f82-e4ed6daf6815" (UID: "4e483809-56dc-44d5-8f82-e4ed6daf6815"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 23:43:47.708052 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:47.707915    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e483809-56dc-44d5-8f82-e4ed6daf6815-istio-token" (OuterVolumeSpecName: "istio-token") pod "4e483809-56dc-44d5-8f82-e4ed6daf6815" (UID: "4e483809-56dc-44d5-8f82-e4ed6daf6815"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:43:47.710358 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:47.710335    2578 generic.go:356] "Generic (PLEG): container finished" podID="4e483809-56dc-44d5-8f82-e4ed6daf6815" containerID="8a4ed1764e2b73abc4ef88963352b1eddbc7de9182014eec3f480b3e1856c5cb" exitCode=0
Jul 23 23:43:47.710462 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:47.710374    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5" event={"ID":"4e483809-56dc-44d5-8f82-e4ed6daf6815","Type":"ContainerDied","Data":"8a4ed1764e2b73abc4ef88963352b1eddbc7de9182014eec3f480b3e1856c5cb"}
Jul 23 23:43:47.710462 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:47.710383    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5"
Jul 23 23:43:47.710462 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:47.710413    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5" event={"ID":"4e483809-56dc-44d5-8f82-e4ed6daf6815","Type":"ContainerDied","Data":"b03d9e9f2f72687f1e080dc2ab78c0c3737d04636fb692f6a391308ead996b6f"}
Jul 23 23:43:47.710462 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:47.710434    2578 scope.go:117] "RemoveContainer" containerID="8a4ed1764e2b73abc4ef88963352b1eddbc7de9182014eec3f480b3e1856c5cb"
Jul 23 23:43:47.726408 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:47.726389    2578 scope.go:117] "RemoveContainer" containerID="b40f072f7b56595475ecbc79fe2f597ce36e5b402ba4ecd891fadd36d4d6f46e"
Jul 23 23:43:47.737788 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:47.737749    2578 scope.go:117] "RemoveContainer" containerID="8a4ed1764e2b73abc4ef88963352b1eddbc7de9182014eec3f480b3e1856c5cb"
Jul 23 23:43:47.738410 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:43:47.738337    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a4ed1764e2b73abc4ef88963352b1eddbc7de9182014eec3f480b3e1856c5cb\": container with ID starting with 8a4ed1764e2b73abc4ef88963352b1eddbc7de9182014eec3f480b3e1856c5cb not found: ID does not exist" containerID="8a4ed1764e2b73abc4ef88963352b1eddbc7de9182014eec3f480b3e1856c5cb"
Jul 23 23:43:47.738643 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:47.738606    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a4ed1764e2b73abc4ef88963352b1eddbc7de9182014eec3f480b3e1856c5cb"} err="failed to get container status \"8a4ed1764e2b73abc4ef88963352b1eddbc7de9182014eec3f480b3e1856c5cb\": rpc error: code = NotFound desc = could not find container \"8a4ed1764e2b73abc4ef88963352b1eddbc7de9182014eec3f480b3e1856c5cb\": container with ID starting with 8a4ed1764e2b73abc4ef88963352b1eddbc7de9182014eec3f480b3e1856c5cb not found: ID does not exist"
Jul 23 23:43:47.738643 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:47.738647    2578 scope.go:117] "RemoveContainer" containerID="b40f072f7b56595475ecbc79fe2f597ce36e5b402ba4ecd891fadd36d4d6f46e"
Jul 23 23:43:47.738880 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:47.738698    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5"]
Jul 23 23:43:47.739209 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:43:47.739189    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b40f072f7b56595475ecbc79fe2f597ce36e5b402ba4ecd891fadd36d4d6f46e\": container with ID starting with b40f072f7b56595475ecbc79fe2f597ce36e5b402ba4ecd891fadd36d4d6f46e not found: ID does not exist" containerID="b40f072f7b56595475ecbc79fe2f597ce36e5b402ba4ecd891fadd36d4d6f46e"
Jul 23 23:43:47.739285 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:47.739219    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b40f072f7b56595475ecbc79fe2f597ce36e5b402ba4ecd891fadd36d4d6f46e"} err="failed to get container status \"b40f072f7b56595475ecbc79fe2f597ce36e5b402ba4ecd891fadd36d4d6f46e\": rpc error: code = NotFound desc = could not find container \"b40f072f7b56595475ecbc79fe2f597ce36e5b402ba4ecd891fadd36d4d6f46e\": container with ID starting with b40f072f7b56595475ecbc79fe2f597ce36e5b402ba4ecd891fadd36d4d6f46e not found: ID does not exist"
Jul 23 23:43:47.740893 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:47.740874    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-4155fe38-openshift-default-cd4d6c548-lq6n5"]
Jul 23 23:43:47.806428 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:47.806390    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e483809-56dc-44d5-8f82-e4ed6daf6815-istio-podinfo\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:43:47.806428 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:47.806421    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e483809-56dc-44d5-8f82-e4ed6daf6815-istio-token\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:43:47.806428 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:47.806432    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e483809-56dc-44d5-8f82-e4ed6daf6815-workload-socket\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:43:47.806680 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:47.806440    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8p4kt\" (UniqueName: \"kubernetes.io/projected/4e483809-56dc-44d5-8f82-e4ed6daf6815-kube-api-access-8p4kt\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:43:47.806680 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:47.806450    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e483809-56dc-44d5-8f82-e4ed6daf6815-istio-data\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:43:47.806680 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:47.806457    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e483809-56dc-44d5-8f82-e4ed6daf6815-istio-envoy\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:43:49.236159 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:43:49.236127    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e483809-56dc-44d5-8f82-e4ed6daf6815" path="/var/lib/kubelet/pods/4e483809-56dc-44d5-8f82-e4ed6daf6815/volumes"
Jul 23 23:44:01.918898 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:01.918862    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck"]
Jul 23 23:44:01.919374 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:01.919242    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f7565a8-5332-4653-8ee7-8d0031962fb4" containerName="maas-api"
Jul 23 23:44:01.919374 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:01.919256    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f7565a8-5332-4653-8ee7-8d0031962fb4" containerName="maas-api"
Jul 23 23:44:01.919374 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:01.919270    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e483809-56dc-44d5-8f82-e4ed6daf6815" containerName="istio-proxy"
Jul 23 23:44:01.919374 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:01.919279    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e483809-56dc-44d5-8f82-e4ed6daf6815" containerName="istio-proxy"
Jul 23 23:44:01.919374 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:01.919301    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e483809-56dc-44d5-8f82-e4ed6daf6815" containerName="istio-proxy"
Jul 23 23:44:01.919374 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:01.919309    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e483809-56dc-44d5-8f82-e4ed6daf6815" containerName="istio-proxy"
Jul 23 23:44:01.919601 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:01.919406    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e483809-56dc-44d5-8f82-e4ed6daf6815" containerName="istio-proxy"
Jul 23 23:44:01.919601 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:01.919416    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f7565a8-5332-4653-8ee7-8d0031962fb4" containerName="maas-api"
Jul 23 23:44:01.919601 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:01.919465    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e483809-56dc-44d5-8f82-e4ed6daf6815" containerName="istio-proxy"
Jul 23 23:44:01.922453 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:01.922432    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck"
Jul 23 23:44:01.925372 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:01.925353    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-88e3c218-openshift-default-dockercfg-2ndqs\""
Jul 23 23:44:01.934279 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:01.934247    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck"]
Jul 23 23:44:02.039722 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:02.039684    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0873af7c-8344-4d23-bed4-3ed9f2461582-workload-certs\") pod \"e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck\" (UID: \"0873af7c-8344-4d23-bed4-3ed9f2461582\") " pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck"
Jul 23 23:44:02.039921 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:02.039738    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxpkn\" (UniqueName: \"kubernetes.io/projected/0873af7c-8344-4d23-bed4-3ed9f2461582-kube-api-access-zxpkn\") pod \"e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck\" (UID: \"0873af7c-8344-4d23-bed4-3ed9f2461582\") " pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck"
Jul 23 23:44:02.039921 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:02.039770    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0873af7c-8344-4d23-bed4-3ed9f2461582-istio-envoy\") pod \"e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck\" (UID: \"0873af7c-8344-4d23-bed4-3ed9f2461582\") " pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck"
Jul 23 23:44:02.039921 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:02.039834    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0873af7c-8344-4d23-bed4-3ed9f2461582-credential-socket\") pod \"e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck\" (UID: \"0873af7c-8344-4d23-bed4-3ed9f2461582\") " pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck"
Jul 23 23:44:02.039921 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:02.039870    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0873af7c-8344-4d23-bed4-3ed9f2461582-istio-token\") pod \"e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck\" (UID: \"0873af7c-8344-4d23-bed4-3ed9f2461582\") " pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck"
Jul 23 23:44:02.039921 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:02.039891    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0873af7c-8344-4d23-bed4-3ed9f2461582-istiod-ca-cert\") pod \"e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck\" (UID: \"0873af7c-8344-4d23-bed4-3ed9f2461582\") " pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck"
Jul 23 23:44:02.040125 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:02.039944    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0873af7c-8344-4d23-bed4-3ed9f2461582-istio-data\") pod \"e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck\" (UID: \"0873af7c-8344-4d23-bed4-3ed9f2461582\") " pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck"
Jul 23 23:44:02.040125 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:02.039963    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0873af7c-8344-4d23-bed4-3ed9f2461582-workload-socket\") pod \"e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck\" (UID: \"0873af7c-8344-4d23-bed4-3ed9f2461582\") " pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck"
Jul 23 23:44:02.040125 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:02.039980    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0873af7c-8344-4d23-bed4-3ed9f2461582-istio-podinfo\") pod \"e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck\" (UID: \"0873af7c-8344-4d23-bed4-3ed9f2461582\") " pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck"
Jul 23 23:44:02.140947 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:02.140900    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxpkn\" (UniqueName: \"kubernetes.io/projected/0873af7c-8344-4d23-bed4-3ed9f2461582-kube-api-access-zxpkn\") pod \"e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck\" (UID: \"0873af7c-8344-4d23-bed4-3ed9f2461582\") " pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck"
Jul 23 23:44:02.141152 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:02.140965    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0873af7c-8344-4d23-bed4-3ed9f2461582-istio-envoy\") pod \"e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck\" (UID: \"0873af7c-8344-4d23-bed4-3ed9f2461582\") " pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck"
Jul 23 23:44:02.141152 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:02.141009    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0873af7c-8344-4d23-bed4-3ed9f2461582-credential-socket\") pod \"e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck\" (UID: \"0873af7c-8344-4d23-bed4-3ed9f2461582\") " pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck"
Jul 23 23:44:02.141152 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:02.141040    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0873af7c-8344-4d23-bed4-3ed9f2461582-istio-token\") pod \"e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck\" (UID: \"0873af7c-8344-4d23-bed4-3ed9f2461582\") " pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck"
Jul 23 23:44:02.141152 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:02.141067    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0873af7c-8344-4d23-bed4-3ed9f2461582-istiod-ca-cert\") pod \"e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck\" (UID: \"0873af7c-8344-4d23-bed4-3ed9f2461582\") " pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck"
Jul 23 23:44:02.141152 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:02.141102    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0873af7c-8344-4d23-bed4-3ed9f2461582-istio-data\") pod \"e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck\" (UID: \"0873af7c-8344-4d23-bed4-3ed9f2461582\") " pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck"
Jul 23 23:44:02.141152 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:02.141128    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0873af7c-8344-4d23-bed4-3ed9f2461582-workload-socket\") pod \"e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck\" (UID: \"0873af7c-8344-4d23-bed4-3ed9f2461582\") " pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck"
Jul 23 23:44:02.141465 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:02.141153    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0873af7c-8344-4d23-bed4-3ed9f2461582-istio-podinfo\") pod \"e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck\" (UID: \"0873af7c-8344-4d23-bed4-3ed9f2461582\") " pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck"
Jul 23 23:44:02.141465 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:02.141246    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0873af7c-8344-4d23-bed4-3ed9f2461582-workload-certs\") pod \"e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck\" (UID: \"0873af7c-8344-4d23-bed4-3ed9f2461582\") " pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck"
Jul 23 23:44:02.141572 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:02.141473    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0873af7c-8344-4d23-bed4-3ed9f2461582-credential-socket\") pod \"e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck\" (UID: \"0873af7c-8344-4d23-bed4-3ed9f2461582\") " pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck"
Jul 23 23:44:02.141572 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:02.141510    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0873af7c-8344-4d23-bed4-3ed9f2461582-istio-data\") pod \"e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck\" (UID: \"0873af7c-8344-4d23-bed4-3ed9f2461582\") " pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck"
Jul 23 23:44:02.141758 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:02.141739    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0873af7c-8344-4d23-bed4-3ed9f2461582-workload-socket\") pod \"e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck\" (UID: \"0873af7c-8344-4d23-bed4-3ed9f2461582\") " pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck"
Jul 23 23:44:02.141813 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:02.141763    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0873af7c-8344-4d23-bed4-3ed9f2461582-workload-certs\") pod \"e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck\" (UID: \"0873af7c-8344-4d23-bed4-3ed9f2461582\") " pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck"
Jul 23 23:44:02.141946 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:02.141923    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0873af7c-8344-4d23-bed4-3ed9f2461582-istiod-ca-cert\") pod \"e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck\" (UID: \"0873af7c-8344-4d23-bed4-3ed9f2461582\") " pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck"
Jul 23 23:44:02.143567 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:02.143542    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0873af7c-8344-4d23-bed4-3ed9f2461582-istio-envoy\") pod \"e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck\" (UID: \"0873af7c-8344-4d23-bed4-3ed9f2461582\") " pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck"
Jul 23 23:44:02.143820 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:02.143802    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0873af7c-8344-4d23-bed4-3ed9f2461582-istio-podinfo\") pod \"e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck\" (UID: \"0873af7c-8344-4d23-bed4-3ed9f2461582\") " pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck"
Jul 23 23:44:02.148911 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:02.148889    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0873af7c-8344-4d23-bed4-3ed9f2461582-istio-token\") pod \"e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck\" (UID: \"0873af7c-8344-4d23-bed4-3ed9f2461582\") " pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck"
Jul 23 23:44:02.149216 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:02.149198    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxpkn\" (UniqueName: \"kubernetes.io/projected/0873af7c-8344-4d23-bed4-3ed9f2461582-kube-api-access-zxpkn\") pod \"e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck\" (UID: \"0873af7c-8344-4d23-bed4-3ed9f2461582\") " pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck"
Jul 23 23:44:02.233585 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:02.233539    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck"
Jul 23 23:44:02.377373 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:02.377343    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck"]
Jul 23 23:44:02.379967 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:44:02.379937    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0873af7c_8344_4d23_bed4_3ed9f2461582.slice/crio-39d62bbde5b6d234f70e8a24db200a83d1375588aa47a17735eb990f2db7aa29 WatchSource:0}: Error finding container 39d62bbde5b6d234f70e8a24db200a83d1375588aa47a17735eb990f2db7aa29: Status 404 returned error can't find the container with id 39d62bbde5b6d234f70e8a24db200a83d1375588aa47a17735eb990f2db7aa29
Jul 23 23:44:02.382271 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:02.382235    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:44:02.382358 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:02.382324    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:44:02.382396 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:02.382364    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:44:02.789539 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:02.789495    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck" event={"ID":"0873af7c-8344-4d23-bed4-3ed9f2461582","Type":"ContainerStarted","Data":"68db0ba3b92be6fb124c1c4fa938f815627bc2483ab03bcb885ea3ec50052271"}
Jul 23 23:44:02.789744 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:02.789595    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck" event={"ID":"0873af7c-8344-4d23-bed4-3ed9f2461582","Type":"ContainerStarted","Data":"39d62bbde5b6d234f70e8a24db200a83d1375588aa47a17735eb990f2db7aa29"}
Jul 23 23:44:02.812730 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:02.812677    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck" podStartSLOduration=1.812646627 podStartE2EDuration="1.812646627s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:44:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:44:02.810157677 +0000 UTC m=+1832.179683339" watchObservedRunningTime="2026-07-23 23:44:02.812646627 +0000 UTC m=+1832.182172286"
Jul 23 23:44:03.236159 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:03.236126    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck"
Jul 23 23:44:03.239195 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:03.239177    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck"
Jul 23 23:44:03.794622 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:03.794590    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck"
Jul 23 23:44:03.795531 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:03.795512    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck"
Jul 23 23:44:04.820770 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:04.820737    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-88e3c218-bcd44f49b-5bsv2"]
Jul 23 23:44:04.826007 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:04.825978    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-88e3c218-bcd44f49b-5bsv2"
Jul 23 23:44:04.828554 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:04.828531    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-88e3c218-serving-cert\""
Jul 23 23:44:04.832316 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:04.832288    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-88e3c218-bcd44f49b-5bsv2"]
Jul 23 23:44:04.971465 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:04.971420    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jw649\" (UniqueName: \"kubernetes.io/projected/70c1875f-7bcf-41be-a11a-bac70c9b2bd9-kube-api-access-jw649\") pod \"maas-api-e2e-mt-88e3c218-bcd44f49b-5bsv2\" (UID: \"70c1875f-7bcf-41be-a11a-bac70c9b2bd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-88e3c218-bcd44f49b-5bsv2"
Jul 23 23:44:04.971680 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:04.971495    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70c1875f-7bcf-41be-a11a-bac70c9b2bd9-maas-api-tls\") pod \"maas-api-e2e-mt-88e3c218-bcd44f49b-5bsv2\" (UID: \"70c1875f-7bcf-41be-a11a-bac70c9b2bd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-88e3c218-bcd44f49b-5bsv2"
Jul 23 23:44:04.971680 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:04.971611    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70c1875f-7bcf-41be-a11a-bac70c9b2bd9-openshift-service-ca\") pod \"maas-api-e2e-mt-88e3c218-bcd44f49b-5bsv2\" (UID: \"70c1875f-7bcf-41be-a11a-bac70c9b2bd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-88e3c218-bcd44f49b-5bsv2"
Jul 23 23:44:05.073110 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:05.073015    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jw649\" (UniqueName: \"kubernetes.io/projected/70c1875f-7bcf-41be-a11a-bac70c9b2bd9-kube-api-access-jw649\") pod \"maas-api-e2e-mt-88e3c218-bcd44f49b-5bsv2\" (UID: \"70c1875f-7bcf-41be-a11a-bac70c9b2bd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-88e3c218-bcd44f49b-5bsv2"
Jul 23 23:44:05.073110 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:05.073074    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70c1875f-7bcf-41be-a11a-bac70c9b2bd9-maas-api-tls\") pod \"maas-api-e2e-mt-88e3c218-bcd44f49b-5bsv2\" (UID: \"70c1875f-7bcf-41be-a11a-bac70c9b2bd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-88e3c218-bcd44f49b-5bsv2"
Jul 23 23:44:05.073110 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:05.073111    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70c1875f-7bcf-41be-a11a-bac70c9b2bd9-openshift-service-ca\") pod \"maas-api-e2e-mt-88e3c218-bcd44f49b-5bsv2\" (UID: \"70c1875f-7bcf-41be-a11a-bac70c9b2bd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-88e3c218-bcd44f49b-5bsv2"
Jul 23 23:44:05.073351 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:44:05.073236    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-88e3c218-serving-cert: secret "maas-api-e2e-mt-88e3c218-serving-cert" not found
Jul 23 23:44:05.073351 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:44:05.073304    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70c1875f-7bcf-41be-a11a-bac70c9b2bd9-maas-api-tls podName:70c1875f-7bcf-41be-a11a-bac70c9b2bd9 nodeName:}" failed. No retries permitted until 2026-07-23 23:44:05.573284401 +0000 UTC m=+1834.942810037 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/70c1875f-7bcf-41be-a11a-bac70c9b2bd9-maas-api-tls") pod "maas-api-e2e-mt-88e3c218-bcd44f49b-5bsv2" (UID: "70c1875f-7bcf-41be-a11a-bac70c9b2bd9") : secret "maas-api-e2e-mt-88e3c218-serving-cert" not found
Jul 23 23:44:05.073808 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:05.073790    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70c1875f-7bcf-41be-a11a-bac70c9b2bd9-openshift-service-ca\") pod \"maas-api-e2e-mt-88e3c218-bcd44f49b-5bsv2\" (UID: \"70c1875f-7bcf-41be-a11a-bac70c9b2bd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-88e3c218-bcd44f49b-5bsv2"
Jul 23 23:44:05.082143 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:05.082110    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jw649\" (UniqueName: \"kubernetes.io/projected/70c1875f-7bcf-41be-a11a-bac70c9b2bd9-kube-api-access-jw649\") pod \"maas-api-e2e-mt-88e3c218-bcd44f49b-5bsv2\" (UID: \"70c1875f-7bcf-41be-a11a-bac70c9b2bd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-88e3c218-bcd44f49b-5bsv2"
Jul 23 23:44:05.578205 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:05.578168    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70c1875f-7bcf-41be-a11a-bac70c9b2bd9-maas-api-tls\") pod \"maas-api-e2e-mt-88e3c218-bcd44f49b-5bsv2\" (UID: \"70c1875f-7bcf-41be-a11a-bac70c9b2bd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-88e3c218-bcd44f49b-5bsv2"
Jul 23 23:44:05.580731 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:05.580699    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70c1875f-7bcf-41be-a11a-bac70c9b2bd9-maas-api-tls\") pod \"maas-api-e2e-mt-88e3c218-bcd44f49b-5bsv2\" (UID: \"70c1875f-7bcf-41be-a11a-bac70c9b2bd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-88e3c218-bcd44f49b-5bsv2"
Jul 23 23:44:05.739095 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:05.739054    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-88e3c218-bcd44f49b-5bsv2"
Jul 23 23:44:05.875105 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:05.875076    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-88e3c218-bcd44f49b-5bsv2"]
Jul 23 23:44:05.877385 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:44:05.877363    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70c1875f_7bcf_41be_a11a_bac70c9b2bd9.slice/crio-300674c24376cbe54ff585bb8081c072c44dc96b33ed496dd2fcdab0e901b1fa WatchSource:0}: Error finding container 300674c24376cbe54ff585bb8081c072c44dc96b33ed496dd2fcdab0e901b1fa: Status 404 returned error can't find the container with id 300674c24376cbe54ff585bb8081c072c44dc96b33ed496dd2fcdab0e901b1fa
Jul 23 23:44:05.878880 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:05.878863    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 23:44:06.812534 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:06.812497    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-88e3c218-bcd44f49b-5bsv2" event={"ID":"70c1875f-7bcf-41be-a11a-bac70c9b2bd9","Type":"ContainerStarted","Data":"09f5bdbbd6b0e47e0289160fb2bd8d835bcdc07b1624ddb5a1810398b4e505d6"}
Jul 23 23:44:06.812785 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:06.812553    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-88e3c218-bcd44f49b-5bsv2" event={"ID":"70c1875f-7bcf-41be-a11a-bac70c9b2bd9","Type":"ContainerStarted","Data":"300674c24376cbe54ff585bb8081c072c44dc96b33ed496dd2fcdab0e901b1fa"}
Jul 23 23:44:06.812785 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:06.812630    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-88e3c218-bcd44f49b-5bsv2"
Jul 23 23:44:06.830089 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:06.830039    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-88e3c218-bcd44f49b-5bsv2" podStartSLOduration=2.511138371 podStartE2EDuration="2.830023889s" totalImagesPullingTime="318.885518ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:44:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:44:06.829244833 +0000 UTC m=+1836.198770492" watchObservedRunningTime="2026-07-23 23:44:06.830023889 +0000 UTC m=+1836.199549548"
Jul 23 23:44:12.823686 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:12.823635    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-88e3c218-bcd44f49b-5bsv2"
Jul 23 23:44:19.592919 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:19.591525    2578 patch_prober.go:28] interesting pod/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": read tcp 10.134.0.2:48610->10.134.0.41:15021: read: connection reset by peer" start-of-body=
Jul 23 23:44:19.592919 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:19.591594    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck" podUID="0873af7c-8344-4d23-bed4-3ed9f2461582" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": read tcp 10.134.0.2:48610->10.134.0.41:15021: read: connection reset by peer"
Jul 23 23:44:19.879617 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:19.879508    2578 generic.go:356] "Generic (PLEG): container finished" podID="0873af7c-8344-4d23-bed4-3ed9f2461582" containerID="68db0ba3b92be6fb124c1c4fa938f815627bc2483ab03bcb885ea3ec50052271" exitCode=0
Jul 23 23:44:19.879617 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:19.879580    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck" event={"ID":"0873af7c-8344-4d23-bed4-3ed9f2461582","Type":"ContainerDied","Data":"68db0ba3b92be6fb124c1c4fa938f815627bc2483ab03bcb885ea3ec50052271"}
Jul 23 23:44:19.880004 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:19.879990    2578 scope.go:117] "RemoveContainer" containerID="68db0ba3b92be6fb124c1c4fa938f815627bc2483ab03bcb885ea3ec50052271"
Jul 23 23:44:19.880973 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:19.880946    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:44:19.881078 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:19.881057    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:44:19.881140 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:19.881111    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:44:20.891254 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:20.891199    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck" event={"ID":"0873af7c-8344-4d23-bed4-3ed9f2461582","Type":"ContainerStarted","Data":"81ef825f33a209962acb3060a461ca705d896f7dc39e8e1d84b666f9ab66b811"}
Jul 23 23:44:21.233960 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:21.233915    2578 patch_prober.go:28] interesting pod/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Jul 23 23:44:21.234160 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:21.233978    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck" podUID="0873af7c-8344-4d23-bed4-3ed9f2461582" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Jul 23 23:44:21.236353 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:21.236333    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck"
Jul 23 23:44:22.233968 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:22.233929    2578 patch_prober.go:28] interesting pod/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Jul 23 23:44:22.234379 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:22.233987    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck" podUID="0873af7c-8344-4d23-bed4-3ed9f2461582" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Jul 23 23:44:23.234397 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:23.234359    2578 patch_prober.go:28] interesting pod/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Jul 23 23:44:23.234814 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:23.234427    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck" podUID="0873af7c-8344-4d23-bed4-3ed9f2461582" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Jul 23 23:44:24.234416 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:24.234379    2578 patch_prober.go:28] interesting pod/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Jul 23 23:44:24.234871 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:24.234435    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck" podUID="0873af7c-8344-4d23-bed4-3ed9f2461582" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Jul 23 23:44:25.237461 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:25.237434    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck"
Jul 23 23:44:25.237906 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:25.237691    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck"
Jul 23 23:44:25.238530 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:25.238510    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck"
Jul 23 23:44:53.002508 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:53.002411    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-88e3c218-bcd44f49b-5bsv2"]
Jul 23 23:44:53.003105 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:53.002707    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-88e3c218-bcd44f49b-5bsv2" podUID="70c1875f-7bcf-41be-a11a-bac70c9b2bd9" containerName="maas-api" containerID="cri-o://09f5bdbbd6b0e47e0289160fb2bd8d835bcdc07b1624ddb5a1810398b4e505d6" gracePeriod=30
Jul 23 23:44:53.254869 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:53.254800    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-88e3c218-bcd44f49b-5bsv2"
Jul 23 23:44:53.354928 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:53.354894    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70c1875f-7bcf-41be-a11a-bac70c9b2bd9-openshift-service-ca\") pod \"70c1875f-7bcf-41be-a11a-bac70c9b2bd9\" (UID: \"70c1875f-7bcf-41be-a11a-bac70c9b2bd9\") "
Jul 23 23:44:53.355103 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:53.354979    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70c1875f-7bcf-41be-a11a-bac70c9b2bd9-maas-api-tls\") pod \"70c1875f-7bcf-41be-a11a-bac70c9b2bd9\" (UID: \"70c1875f-7bcf-41be-a11a-bac70c9b2bd9\") "
Jul 23 23:44:53.355201 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:53.355165    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jw649\" (UniqueName: \"kubernetes.io/projected/70c1875f-7bcf-41be-a11a-bac70c9b2bd9-kube-api-access-jw649\") pod \"70c1875f-7bcf-41be-a11a-bac70c9b2bd9\" (UID: \"70c1875f-7bcf-41be-a11a-bac70c9b2bd9\") "
Jul 23 23:44:53.355350 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:53.355324    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70c1875f-7bcf-41be-a11a-bac70c9b2bd9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "70c1875f-7bcf-41be-a11a-bac70c9b2bd9" (UID: "70c1875f-7bcf-41be-a11a-bac70c9b2bd9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:44:53.355563 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:53.355544    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70c1875f-7bcf-41be-a11a-bac70c9b2bd9-openshift-service-ca\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:44:53.357360 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:53.357328    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70c1875f-7bcf-41be-a11a-bac70c9b2bd9-kube-api-access-jw649" (OuterVolumeSpecName: "kube-api-access-jw649") pod "70c1875f-7bcf-41be-a11a-bac70c9b2bd9" (UID: "70c1875f-7bcf-41be-a11a-bac70c9b2bd9"). InnerVolumeSpecName "kube-api-access-jw649". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:44:53.357458 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:53.357377    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70c1875f-7bcf-41be-a11a-bac70c9b2bd9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "70c1875f-7bcf-41be-a11a-bac70c9b2bd9" (UID: "70c1875f-7bcf-41be-a11a-bac70c9b2bd9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 23:44:53.456771 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:53.456715    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70c1875f-7bcf-41be-a11a-bac70c9b2bd9-maas-api-tls\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:44:53.456771 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:53.456768    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jw649\" (UniqueName: \"kubernetes.io/projected/70c1875f-7bcf-41be-a11a-bac70c9b2bd9-kube-api-access-jw649\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:44:54.054095 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:54.054055    2578 generic.go:356] "Generic (PLEG): container finished" podID="70c1875f-7bcf-41be-a11a-bac70c9b2bd9" containerID="09f5bdbbd6b0e47e0289160fb2bd8d835bcdc07b1624ddb5a1810398b4e505d6" exitCode=0
Jul 23 23:44:54.054564 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:54.054104    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-88e3c218-bcd44f49b-5bsv2"
Jul 23 23:44:54.054564 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:54.054147    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-88e3c218-bcd44f49b-5bsv2" event={"ID":"70c1875f-7bcf-41be-a11a-bac70c9b2bd9","Type":"ContainerDied","Data":"09f5bdbbd6b0e47e0289160fb2bd8d835bcdc07b1624ddb5a1810398b4e505d6"}
Jul 23 23:44:54.054564 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:54.054215    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-88e3c218-bcd44f49b-5bsv2" event={"ID":"70c1875f-7bcf-41be-a11a-bac70c9b2bd9","Type":"ContainerDied","Data":"300674c24376cbe54ff585bb8081c072c44dc96b33ed496dd2fcdab0e901b1fa"}
Jul 23 23:44:54.054564 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:54.054251    2578 scope.go:117] "RemoveContainer" containerID="09f5bdbbd6b0e47e0289160fb2bd8d835bcdc07b1624ddb5a1810398b4e505d6"
Jul 23 23:44:54.063943 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:54.063926    2578 scope.go:117] "RemoveContainer" containerID="09f5bdbbd6b0e47e0289160fb2bd8d835bcdc07b1624ddb5a1810398b4e505d6"
Jul 23 23:44:54.064202 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:44:54.064184    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09f5bdbbd6b0e47e0289160fb2bd8d835bcdc07b1624ddb5a1810398b4e505d6\": container with ID starting with 09f5bdbbd6b0e47e0289160fb2bd8d835bcdc07b1624ddb5a1810398b4e505d6 not found: ID does not exist" containerID="09f5bdbbd6b0e47e0289160fb2bd8d835bcdc07b1624ddb5a1810398b4e505d6"
Jul 23 23:44:54.064243 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:54.064210    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09f5bdbbd6b0e47e0289160fb2bd8d835bcdc07b1624ddb5a1810398b4e505d6"} err="failed to get container status \"09f5bdbbd6b0e47e0289160fb2bd8d835bcdc07b1624ddb5a1810398b4e505d6\": rpc error: code = NotFound desc = could not find container \"09f5bdbbd6b0e47e0289160fb2bd8d835bcdc07b1624ddb5a1810398b4e505d6\": container with ID starting with 09f5bdbbd6b0e47e0289160fb2bd8d835bcdc07b1624ddb5a1810398b4e505d6 not found: ID does not exist"
Jul 23 23:44:54.076071 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:54.076037    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-88e3c218-bcd44f49b-5bsv2"]
Jul 23 23:44:54.078798 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:54.078773    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-88e3c218-bcd44f49b-5bsv2"]
Jul 23 23:44:55.236256 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:44:55.236222    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70c1875f-7bcf-41be-a11a-bac70c9b2bd9" path="/var/lib/kubelet/pods/70c1875f-7bcf-41be-a11a-bac70c9b2bd9/volumes"
Jul 23 23:45:00.133062 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:00.133016    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747505-bgd7q"]
Jul 23 23:45:00.133583 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:00.133561    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70c1875f-7bcf-41be-a11a-bac70c9b2bd9" containerName="maas-api"
Jul 23 23:45:00.133698 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:00.133586    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="70c1875f-7bcf-41be-a11a-bac70c9b2bd9" containerName="maas-api"
Jul 23 23:45:00.133762 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:00.133736    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="70c1875f-7bcf-41be-a11a-bac70c9b2bd9" containerName="maas-api"
Jul 23 23:45:00.137947 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:00.137925    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747505-bgd7q"
Jul 23 23:45:00.140956 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:00.140931    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lzd6x\""
Jul 23 23:45:00.142615 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:00.142590    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747505-bgd7q"]
Jul 23 23:45:00.149295 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:00.149277    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747505-bgd7q"
Jul 23 23:45:00.289712 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:00.289683    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747505-bgd7q"]
Jul 23 23:45:00.292543 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:45:00.292507    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1725205b_76b9_4902_989f_50ed2e38a3f3.slice/crio-f52b7970c844672a7101c551b07f0f8b8882db85dddce23b1f09fdd1051d4c21 WatchSource:0}: Error finding container f52b7970c844672a7101c551b07f0f8b8882db85dddce23b1f09fdd1051d4c21: Status 404 returned error can't find the container with id f52b7970c844672a7101c551b07f0f8b8882db85dddce23b1f09fdd1051d4c21
Jul 23 23:45:01.089078 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:01.089032    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747505-bgd7q" event={"ID":"1725205b-76b9-4902-989f-50ed2e38a3f3","Type":"ContainerStarted","Data":"0a7ef1f0c98f06784a0f7bf04dc068d60090533f26a11f28de536eb264449711"}
Jul 23 23:45:01.089254 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:01.089140    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747505-bgd7q" event={"ID":"1725205b-76b9-4902-989f-50ed2e38a3f3","Type":"ContainerStarted","Data":"f52b7970c844672a7101c551b07f0f8b8882db85dddce23b1f09fdd1051d4c21"}
Jul 23 23:45:01.104189 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:01.104146    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747505-bgd7q" podStartSLOduration=1.104132365 podStartE2EDuration="1.104132365s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:45:01.102375281 +0000 UTC m=+1890.471900943" watchObservedRunningTime="2026-07-23 23:45:01.104132365 +0000 UTC m=+1890.473658043"
Jul 23 23:45:02.095489 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:02.095456    2578 generic.go:356] "Generic (PLEG): container finished" podID="1725205b-76b9-4902-989f-50ed2e38a3f3" containerID="0a7ef1f0c98f06784a0f7bf04dc068d60090533f26a11f28de536eb264449711" exitCode=0
Jul 23 23:45:02.095905 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:02.095520    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747505-bgd7q" event={"ID":"1725205b-76b9-4902-989f-50ed2e38a3f3","Type":"ContainerDied","Data":"0a7ef1f0c98f06784a0f7bf04dc068d60090533f26a11f28de536eb264449711"}
Jul 23 23:45:03.235059 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:03.235035    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747505-bgd7q"
Jul 23 23:45:04.105951 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:04.105916    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747505-bgd7q"
Jul 23 23:45:04.105951 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:04.105936    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747505-bgd7q" event={"ID":"1725205b-76b9-4902-989f-50ed2e38a3f3","Type":"ContainerDied","Data":"f52b7970c844672a7101c551b07f0f8b8882db85dddce23b1f09fdd1051d4c21"}
Jul 23 23:45:04.106173 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:04.105987    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f52b7970c844672a7101c551b07f0f8b8882db85dddce23b1f09fdd1051d4c21"
Jul 23 23:45:48.063969 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:48.063924    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck"]
Jul 23 23:45:48.064479 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:48.064280    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck" podUID="0873af7c-8344-4d23-bed4-3ed9f2461582" containerName="istio-proxy" containerID="cri-o://81ef825f33a209962acb3060a461ca705d896f7dc39e8e1d84b666f9ab66b811" gracePeriod=30
Jul 23 23:45:53.326831 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:53.326795    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck"
Jul 23 23:45:53.345987 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:53.345898    2578 generic.go:356] "Generic (PLEG): container finished" podID="0873af7c-8344-4d23-bed4-3ed9f2461582" containerID="81ef825f33a209962acb3060a461ca705d896f7dc39e8e1d84b666f9ab66b811" exitCode=0
Jul 23 23:45:53.345987 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:53.345971    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck"
Jul 23 23:45:53.346185 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:53.345984    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck" event={"ID":"0873af7c-8344-4d23-bed4-3ed9f2461582","Type":"ContainerDied","Data":"81ef825f33a209962acb3060a461ca705d896f7dc39e8e1d84b666f9ab66b811"}
Jul 23 23:45:53.346185 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:53.346078    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck" event={"ID":"0873af7c-8344-4d23-bed4-3ed9f2461582","Type":"ContainerDied","Data":"39d62bbde5b6d234f70e8a24db200a83d1375588aa47a17735eb990f2db7aa29"}
Jul 23 23:45:53.346185 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:53.346109    2578 scope.go:117] "RemoveContainer" containerID="81ef825f33a209962acb3060a461ca705d896f7dc39e8e1d84b666f9ab66b811"
Jul 23 23:45:53.358300 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:53.358276    2578 scope.go:117] "RemoveContainer" containerID="68db0ba3b92be6fb124c1c4fa938f815627bc2483ab03bcb885ea3ec50052271"
Jul 23 23:45:53.367047 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:53.367005    2578 scope.go:117] "RemoveContainer" containerID="81ef825f33a209962acb3060a461ca705d896f7dc39e8e1d84b666f9ab66b811"
Jul 23 23:45:53.367368 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:45:53.367342    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81ef825f33a209962acb3060a461ca705d896f7dc39e8e1d84b666f9ab66b811\": container with ID starting with 81ef825f33a209962acb3060a461ca705d896f7dc39e8e1d84b666f9ab66b811 not found: ID does not exist" containerID="81ef825f33a209962acb3060a461ca705d896f7dc39e8e1d84b666f9ab66b811"
Jul 23 23:45:53.367461 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:53.367376    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81ef825f33a209962acb3060a461ca705d896f7dc39e8e1d84b666f9ab66b811"} err="failed to get container status \"81ef825f33a209962acb3060a461ca705d896f7dc39e8e1d84b666f9ab66b811\": rpc error: code = NotFound desc = could not find container \"81ef825f33a209962acb3060a461ca705d896f7dc39e8e1d84b666f9ab66b811\": container with ID starting with 81ef825f33a209962acb3060a461ca705d896f7dc39e8e1d84b666f9ab66b811 not found: ID does not exist"
Jul 23 23:45:53.367461 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:53.367396    2578 scope.go:117] "RemoveContainer" containerID="68db0ba3b92be6fb124c1c4fa938f815627bc2483ab03bcb885ea3ec50052271"
Jul 23 23:45:53.367714 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:45:53.367691    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68db0ba3b92be6fb124c1c4fa938f815627bc2483ab03bcb885ea3ec50052271\": container with ID starting with 68db0ba3b92be6fb124c1c4fa938f815627bc2483ab03bcb885ea3ec50052271 not found: ID does not exist" containerID="68db0ba3b92be6fb124c1c4fa938f815627bc2483ab03bcb885ea3ec50052271"
Jul 23 23:45:53.367787 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:53.367720    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68db0ba3b92be6fb124c1c4fa938f815627bc2483ab03bcb885ea3ec50052271"} err="failed to get container status \"68db0ba3b92be6fb124c1c4fa938f815627bc2483ab03bcb885ea3ec50052271\": rpc error: code = NotFound desc = could not find container \"68db0ba3b92be6fb124c1c4fa938f815627bc2483ab03bcb885ea3ec50052271\": container with ID starting with 68db0ba3b92be6fb124c1c4fa938f815627bc2483ab03bcb885ea3ec50052271 not found: ID does not exist"
Jul 23 23:45:53.420386 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:53.420342    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0873af7c-8344-4d23-bed4-3ed9f2461582-istio-envoy\") pod \"0873af7c-8344-4d23-bed4-3ed9f2461582\" (UID: \"0873af7c-8344-4d23-bed4-3ed9f2461582\") "
Jul 23 23:45:53.420386 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:53.420390    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0873af7c-8344-4d23-bed4-3ed9f2461582-credential-socket\") pod \"0873af7c-8344-4d23-bed4-3ed9f2461582\" (UID: \"0873af7c-8344-4d23-bed4-3ed9f2461582\") "
Jul 23 23:45:53.420615 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:53.420466    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0873af7c-8344-4d23-bed4-3ed9f2461582-istiod-ca-cert\") pod \"0873af7c-8344-4d23-bed4-3ed9f2461582\" (UID: \"0873af7c-8344-4d23-bed4-3ed9f2461582\") "
Jul 23 23:45:53.420615 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:53.420501    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zxpkn\" (UniqueName: \"kubernetes.io/projected/0873af7c-8344-4d23-bed4-3ed9f2461582-kube-api-access-zxpkn\") pod \"0873af7c-8344-4d23-bed4-3ed9f2461582\" (UID: \"0873af7c-8344-4d23-bed4-3ed9f2461582\") "
Jul 23 23:45:53.420615 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:53.420518    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0873af7c-8344-4d23-bed4-3ed9f2461582-istio-token\") pod \"0873af7c-8344-4d23-bed4-3ed9f2461582\" (UID: \"0873af7c-8344-4d23-bed4-3ed9f2461582\") "
Jul 23 23:45:53.420615 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:53.420536    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0873af7c-8344-4d23-bed4-3ed9f2461582-workload-certs\") pod \"0873af7c-8344-4d23-bed4-3ed9f2461582\" (UID: \"0873af7c-8344-4d23-bed4-3ed9f2461582\") "
Jul 23 23:45:53.420615 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:53.420551    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0873af7c-8344-4d23-bed4-3ed9f2461582-istio-data\") pod \"0873af7c-8344-4d23-bed4-3ed9f2461582\" (UID: \"0873af7c-8344-4d23-bed4-3ed9f2461582\") "
Jul 23 23:45:53.420615 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:53.420574    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0873af7c-8344-4d23-bed4-3ed9f2461582-workload-socket\") pod \"0873af7c-8344-4d23-bed4-3ed9f2461582\" (UID: \"0873af7c-8344-4d23-bed4-3ed9f2461582\") "
Jul 23 23:45:53.420931 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:53.420623    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0873af7c-8344-4d23-bed4-3ed9f2461582-istio-podinfo\") pod \"0873af7c-8344-4d23-bed4-3ed9f2461582\" (UID: \"0873af7c-8344-4d23-bed4-3ed9f2461582\") "
Jul 23 23:45:53.420988 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:53.420931    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0873af7c-8344-4d23-bed4-3ed9f2461582-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0873af7c-8344-4d23-bed4-3ed9f2461582" (UID: "0873af7c-8344-4d23-bed4-3ed9f2461582"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:45:53.420988 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:53.420916    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0873af7c-8344-4d23-bed4-3ed9f2461582-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0873af7c-8344-4d23-bed4-3ed9f2461582" (UID: "0873af7c-8344-4d23-bed4-3ed9f2461582"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:45:53.421217 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:53.421194    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0873af7c-8344-4d23-bed4-3ed9f2461582-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0873af7c-8344-4d23-bed4-3ed9f2461582" (UID: "0873af7c-8344-4d23-bed4-3ed9f2461582"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:45:53.421343 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:53.421089    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0873af7c-8344-4d23-bed4-3ed9f2461582-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0873af7c-8344-4d23-bed4-3ed9f2461582" (UID: "0873af7c-8344-4d23-bed4-3ed9f2461582"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:45:53.421529 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:53.421489    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0873af7c-8344-4d23-bed4-3ed9f2461582-istio-data" (OuterVolumeSpecName: "istio-data") pod "0873af7c-8344-4d23-bed4-3ed9f2461582" (UID: "0873af7c-8344-4d23-bed4-3ed9f2461582"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:45:53.423136 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:53.423111    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0873af7c-8344-4d23-bed4-3ed9f2461582-istio-token" (OuterVolumeSpecName: "istio-token") pod "0873af7c-8344-4d23-bed4-3ed9f2461582" (UID: "0873af7c-8344-4d23-bed4-3ed9f2461582"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:45:53.423234 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:53.423135    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0873af7c-8344-4d23-bed4-3ed9f2461582-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0873af7c-8344-4d23-bed4-3ed9f2461582" (UID: "0873af7c-8344-4d23-bed4-3ed9f2461582"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 23:45:53.423500 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:53.423476    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0873af7c-8344-4d23-bed4-3ed9f2461582-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0873af7c-8344-4d23-bed4-3ed9f2461582" (UID: "0873af7c-8344-4d23-bed4-3ed9f2461582"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:45:53.423561 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:53.423478    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0873af7c-8344-4d23-bed4-3ed9f2461582-kube-api-access-zxpkn" (OuterVolumeSpecName: "kube-api-access-zxpkn") pod "0873af7c-8344-4d23-bed4-3ed9f2461582" (UID: "0873af7c-8344-4d23-bed4-3ed9f2461582"). InnerVolumeSpecName "kube-api-access-zxpkn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:45:53.521444 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:53.521402    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0873af7c-8344-4d23-bed4-3ed9f2461582-istio-data\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:45:53.521444 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:53.521437    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0873af7c-8344-4d23-bed4-3ed9f2461582-workload-socket\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:45:53.521444 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:53.521447    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0873af7c-8344-4d23-bed4-3ed9f2461582-istio-podinfo\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:45:53.521444 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:53.521456    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0873af7c-8344-4d23-bed4-3ed9f2461582-istio-envoy\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:45:53.521759 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:53.521465    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0873af7c-8344-4d23-bed4-3ed9f2461582-credential-socket\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:45:53.521759 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:53.521474    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0873af7c-8344-4d23-bed4-3ed9f2461582-istiod-ca-cert\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:45:53.521759 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:53.521482    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zxpkn\" (UniqueName: \"kubernetes.io/projected/0873af7c-8344-4d23-bed4-3ed9f2461582-kube-api-access-zxpkn\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:45:53.521759 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:53.521490    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0873af7c-8344-4d23-bed4-3ed9f2461582-istio-token\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:45:53.521759 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:53.521498    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0873af7c-8344-4d23-bed4-3ed9f2461582-workload-certs\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:45:53.670684 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:53.670632    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck"]
Jul 23 23:45:53.674118 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:53.674085    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-88e3c218-openshift-default-bb64fd9cb-rnwck"]
Jul 23 23:45:55.236771 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:45:55.236725    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0873af7c-8344-4d23-bed4-3ed9f2461582" path="/var/lib/kubelet/pods/0873af7c-8344-4d23-bed4-3ed9f2461582/volumes"
Jul 23 23:46:43.687392 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:43.687357    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"]
Jul 23 23:46:43.687982 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:43.687936    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0873af7c-8344-4d23-bed4-3ed9f2461582" containerName="istio-proxy"
Jul 23 23:46:43.687982 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:43.687956    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0873af7c-8344-4d23-bed4-3ed9f2461582" containerName="istio-proxy"
Jul 23 23:46:43.688104 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:43.687987    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1725205b-76b9-4902-989f-50ed2e38a3f3" containerName="cleanup"
Jul 23 23:46:43.688104 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:43.687996    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1725205b-76b9-4902-989f-50ed2e38a3f3" containerName="cleanup"
Jul 23 23:46:43.688104 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:43.688077    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0873af7c-8344-4d23-bed4-3ed9f2461582" containerName="istio-proxy"
Jul 23 23:46:43.688104 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:43.688094    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1725205b-76b9-4902-989f-50ed2e38a3f3" containerName="cleanup"
Jul 23 23:46:43.691215 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:43.691194    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"
Jul 23 23:46:43.693869 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:43.693845    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-c228ab-openshift-default-dockercfg-jjbqm\""
Jul 23 23:46:43.701898 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:43.701871    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"]
Jul 23 23:46:43.804508 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:43.804466    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-workload-certs\") pod \"e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw\" (UID: \"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea\") " pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"
Jul 23 23:46:43.804508 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:43.804507    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-credential-socket\") pod \"e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw\" (UID: \"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea\") " pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"
Jul 23 23:46:43.804755 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:43.804553    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-istio-envoy\") pod \"e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw\" (UID: \"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea\") " pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"
Jul 23 23:46:43.804755 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:43.804585    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-workload-socket\") pod \"e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw\" (UID: \"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea\") " pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"
Jul 23 23:46:43.804755 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:43.804617    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-istio-token\") pod \"e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw\" (UID: \"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea\") " pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"
Jul 23 23:46:43.804755 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:43.804642    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbzp4\" (UniqueName: \"kubernetes.io/projected/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-kube-api-access-rbzp4\") pod \"e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw\" (UID: \"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea\") " pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"
Jul 23 23:46:43.804755 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:43.804692    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-istiod-ca-cert\") pod \"e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw\" (UID: \"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea\") " pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"
Jul 23 23:46:43.804921 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:43.804749    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-istio-data\") pod \"e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw\" (UID: \"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea\") " pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"
Jul 23 23:46:43.804921 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:43.804793    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-istio-podinfo\") pod \"e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw\" (UID: \"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea\") " pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"
Jul 23 23:46:43.906276 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:43.906238    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-istiod-ca-cert\") pod \"e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw\" (UID: \"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea\") " pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"
Jul 23 23:46:43.906276 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:43.906286    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-istio-data\") pod \"e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw\" (UID: \"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea\") " pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"
Jul 23 23:46:43.906537 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:43.906327    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-istio-podinfo\") pod \"e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw\" (UID: \"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea\") " pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"
Jul 23 23:46:43.906537 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:43.906377    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-workload-certs\") pod \"e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw\" (UID: \"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea\") " pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"
Jul 23 23:46:43.906537 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:43.906496    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-credential-socket\") pod \"e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw\" (UID: \"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea\") " pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"
Jul 23 23:46:43.906762 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:43.906577    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-istio-envoy\") pod \"e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw\" (UID: \"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea\") " pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"
Jul 23 23:46:43.906762 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:43.906607    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-workload-socket\") pod \"e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw\" (UID: \"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea\") " pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"
Jul 23 23:46:43.906762 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:43.906674    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-istio-token\") pod \"e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw\" (UID: \"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea\") " pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"
Jul 23 23:46:43.906762 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:43.906720    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbzp4\" (UniqueName: \"kubernetes.io/projected/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-kube-api-access-rbzp4\") pod \"e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw\" (UID: \"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea\") " pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"
Jul 23 23:46:43.906973 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:43.906844    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-istio-data\") pod \"e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw\" (UID: \"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea\") " pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"
Jul 23 23:46:43.906973 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:43.906865    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-workload-certs\") pod \"e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw\" (UID: \"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea\") " pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"
Jul 23 23:46:43.906973 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:43.906902    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-credential-socket\") pod \"e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw\" (UID: \"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea\") " pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"
Jul 23 23:46:43.907154 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:43.907131    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-workload-socket\") pod \"e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw\" (UID: \"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea\") " pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"
Jul 23 23:46:43.907775 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:43.907748    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-istiod-ca-cert\") pod \"e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw\" (UID: \"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea\") " pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"
Jul 23 23:46:43.910397 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:43.910370    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-istio-podinfo\") pod \"e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw\" (UID: \"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea\") " pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"
Jul 23 23:46:43.911473 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:43.911457    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-istio-envoy\") pod \"e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw\" (UID: \"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea\") " pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"
Jul 23 23:46:43.914553 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:43.914529    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-istio-token\") pod \"e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw\" (UID: \"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea\") " pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"
Jul 23 23:46:43.915033 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:43.915005    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbzp4\" (UniqueName: \"kubernetes.io/projected/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-kube-api-access-rbzp4\") pod \"e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw\" (UID: \"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea\") " pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"
Jul 23 23:46:44.003948 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:44.003860    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"
Jul 23 23:46:44.139318 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:44.139290    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"]
Jul 23 23:46:44.141578 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:46:44.141548    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5492bdc3_62e0_43c9_b6e9_cb334db3a3ea.slice/crio-5d3bcb9a34cde387cb53ea69e43368d5f6dad0e650196fd367a4ab2dec4766a5 WatchSource:0}: Error finding container 5d3bcb9a34cde387cb53ea69e43368d5f6dad0e650196fd367a4ab2dec4766a5: Status 404 returned error can't find the container with id 5d3bcb9a34cde387cb53ea69e43368d5f6dad0e650196fd367a4ab2dec4766a5
Jul 23 23:46:44.143716 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:44.143686    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:46:44.143798 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:44.143750    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:46:44.143798 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:44.143777    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:46:44.601988 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:44.601952    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw" event={"ID":"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea","Type":"ContainerStarted","Data":"16fbb97be2754d42445a5633a5bd5c1af291a9c9ebf0717c56799f7376c8b480"}
Jul 23 23:46:44.602214 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:44.602004    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw" event={"ID":"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea","Type":"ContainerStarted","Data":"5d3bcb9a34cde387cb53ea69e43368d5f6dad0e650196fd367a4ab2dec4766a5"}
Jul 23 23:46:44.627840 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:44.627773    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw" podStartSLOduration=1.627757747 podStartE2EDuration="1.627757747s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:46:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:46:44.625254809 +0000 UTC m=+1993.994780469" watchObservedRunningTime="2026-07-23 23:46:44.627757747 +0000 UTC m=+1993.997283406"
Jul 23 23:46:45.004827 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:45.004786    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"
Jul 23 23:46:45.009805 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:45.009780    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"
Jul 23 23:46:45.606711 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:45.606671    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"
Jul 23 23:46:45.607861 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:45.607829    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"
Jul 23 23:46:46.677496 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:46.677464    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c228ab-86cd756c6-86pgt"]
Jul 23 23:46:46.678000 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:46.677935    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0873af7c-8344-4d23-bed4-3ed9f2461582" containerName="istio-proxy"
Jul 23 23:46:46.678000 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:46.677949    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0873af7c-8344-4d23-bed4-3ed9f2461582" containerName="istio-proxy"
Jul 23 23:46:46.678116 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:46.678013    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0873af7c-8344-4d23-bed4-3ed9f2461582" containerName="istio-proxy"
Jul 23 23:46:46.681164 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:46.681143    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c228ab-86cd756c6-86pgt"
Jul 23 23:46:46.683741 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:46.683720    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-c228ab-serving-cert\""
Jul 23 23:46:46.690628 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:46.690608    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c228ab-86cd756c6-86pgt"]
Jul 23 23:46:46.833573 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:46.833530    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/290afd93-a50f-441b-a5f0-25170ec4f048-openshift-service-ca\") pod \"maas-api-e2e-inf-a-c228ab-86cd756c6-86pgt\" (UID: \"290afd93-a50f-441b-a5f0-25170ec4f048\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c228ab-86cd756c6-86pgt"
Jul 23 23:46:46.833790 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:46.833624    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/290afd93-a50f-441b-a5f0-25170ec4f048-maas-api-tls\") pod \"maas-api-e2e-inf-a-c228ab-86cd756c6-86pgt\" (UID: \"290afd93-a50f-441b-a5f0-25170ec4f048\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c228ab-86cd756c6-86pgt"
Jul 23 23:46:46.833790 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:46.833737    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2lxb\" (UniqueName: \"kubernetes.io/projected/290afd93-a50f-441b-a5f0-25170ec4f048-kube-api-access-q2lxb\") pod \"maas-api-e2e-inf-a-c228ab-86cd756c6-86pgt\" (UID: \"290afd93-a50f-441b-a5f0-25170ec4f048\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c228ab-86cd756c6-86pgt"
Jul 23 23:46:46.934616 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:46.934522    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/290afd93-a50f-441b-a5f0-25170ec4f048-maas-api-tls\") pod \"maas-api-e2e-inf-a-c228ab-86cd756c6-86pgt\" (UID: \"290afd93-a50f-441b-a5f0-25170ec4f048\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c228ab-86cd756c6-86pgt"
Jul 23 23:46:46.934616 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:46.934573    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2lxb\" (UniqueName: \"kubernetes.io/projected/290afd93-a50f-441b-a5f0-25170ec4f048-kube-api-access-q2lxb\") pod \"maas-api-e2e-inf-a-c228ab-86cd756c6-86pgt\" (UID: \"290afd93-a50f-441b-a5f0-25170ec4f048\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c228ab-86cd756c6-86pgt"
Jul 23 23:46:46.934831 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:46.934620    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/290afd93-a50f-441b-a5f0-25170ec4f048-openshift-service-ca\") pod \"maas-api-e2e-inf-a-c228ab-86cd756c6-86pgt\" (UID: \"290afd93-a50f-441b-a5f0-25170ec4f048\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c228ab-86cd756c6-86pgt"
Jul 23 23:46:46.934831 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:46:46.934690    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-c228ab-serving-cert: secret "maas-api-e2e-inf-a-c228ab-serving-cert" not found
Jul 23 23:46:46.934831 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:46:46.934774    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/290afd93-a50f-441b-a5f0-25170ec4f048-maas-api-tls podName:290afd93-a50f-441b-a5f0-25170ec4f048 nodeName:}" failed. No retries permitted until 2026-07-23 23:46:47.434755894 +0000 UTC m=+1996.804281531 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/290afd93-a50f-441b-a5f0-25170ec4f048-maas-api-tls") pod "maas-api-e2e-inf-a-c228ab-86cd756c6-86pgt" (UID: "290afd93-a50f-441b-a5f0-25170ec4f048") : secret "maas-api-e2e-inf-a-c228ab-serving-cert" not found
Jul 23 23:46:46.935301 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:46.935283    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/290afd93-a50f-441b-a5f0-25170ec4f048-openshift-service-ca\") pod \"maas-api-e2e-inf-a-c228ab-86cd756c6-86pgt\" (UID: \"290afd93-a50f-441b-a5f0-25170ec4f048\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c228ab-86cd756c6-86pgt"
Jul 23 23:46:46.943445 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:46.943414    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2lxb\" (UniqueName: \"kubernetes.io/projected/290afd93-a50f-441b-a5f0-25170ec4f048-kube-api-access-q2lxb\") pod \"maas-api-e2e-inf-a-c228ab-86cd756c6-86pgt\" (UID: \"290afd93-a50f-441b-a5f0-25170ec4f048\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c228ab-86cd756c6-86pgt"
Jul 23 23:46:47.440811 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:47.440771    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/290afd93-a50f-441b-a5f0-25170ec4f048-maas-api-tls\") pod \"maas-api-e2e-inf-a-c228ab-86cd756c6-86pgt\" (UID: \"290afd93-a50f-441b-a5f0-25170ec4f048\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c228ab-86cd756c6-86pgt"
Jul 23 23:46:47.443327 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:47.443296    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/290afd93-a50f-441b-a5f0-25170ec4f048-maas-api-tls\") pod \"maas-api-e2e-inf-a-c228ab-86cd756c6-86pgt\" (UID: \"290afd93-a50f-441b-a5f0-25170ec4f048\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c228ab-86cd756c6-86pgt"
Jul 23 23:46:47.594155 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:47.594110    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c228ab-86cd756c6-86pgt"
Jul 23 23:46:47.760799 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:47.760774    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c228ab-86cd756c6-86pgt"]
Jul 23 23:46:47.762260 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:46:47.762235    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod290afd93_a50f_441b_a5f0_25170ec4f048.slice/crio-13c0f68c20664b77eedab1b21948c8fd501d1e3d4701c2a5bbe84591202ff4db WatchSource:0}: Error finding container 13c0f68c20664b77eedab1b21948c8fd501d1e3d4701c2a5bbe84591202ff4db: Status 404 returned error can't find the container with id 13c0f68c20664b77eedab1b21948c8fd501d1e3d4701c2a5bbe84591202ff4db
Jul 23 23:46:48.624058 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:48.624017    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c228ab-86cd756c6-86pgt" event={"ID":"290afd93-a50f-441b-a5f0-25170ec4f048","Type":"ContainerStarted","Data":"52ac49d456d63aa677c477d1928fad571426ec6bfdef446e733a8b61563de24f"}
Jul 23 23:46:48.624271 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:48.624081    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c228ab-86cd756c6-86pgt" event={"ID":"290afd93-a50f-441b-a5f0-25170ec4f048","Type":"ContainerStarted","Data":"13c0f68c20664b77eedab1b21948c8fd501d1e3d4701c2a5bbe84591202ff4db"}
Jul 23 23:46:48.624271 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:48.624249    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c228ab-86cd756c6-86pgt"
Jul 23 23:46:48.643018 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:48.642962    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c228ab-86cd756c6-86pgt" podStartSLOduration=2.26405946 podStartE2EDuration="2.642948211s" totalImagesPullingTime="378.888751ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:46:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:46:48.641384079 +0000 UTC m=+1998.010909739" watchObservedRunningTime="2026-07-23 23:46:48.642948211 +0000 UTC m=+1998.012473934"
Jul 23 23:46:51.745550 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:51.745516    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"]
Jul 23 23:46:51.749239 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:51.749219    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"
Jul 23 23:46:51.752905 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:51.752883    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-d7e557-openshift-default-dockercfg-dcvph\""
Jul 23 23:46:51.761051 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:51.761028    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"]
Jul 23 23:46:51.884573 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:51.884533    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-istio-token\") pod \"e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22\" (UID: \"5ac35d4e-3dcf-4b95-80c8-90348e8c816a\") " pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"
Jul 23 23:46:51.884788 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:51.884580    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bngh\" (UniqueName: \"kubernetes.io/projected/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-kube-api-access-2bngh\") pod \"e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22\" (UID: \"5ac35d4e-3dcf-4b95-80c8-90348e8c816a\") " pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"
Jul 23 23:46:51.884788 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:51.884675    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-istio-podinfo\") pod \"e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22\" (UID: \"5ac35d4e-3dcf-4b95-80c8-90348e8c816a\") " pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"
Jul 23 23:46:51.884897 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:51.884784    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-istiod-ca-cert\") pod \"e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22\" (UID: \"5ac35d4e-3dcf-4b95-80c8-90348e8c816a\") " pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"
Jul 23 23:46:51.884897 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:51.884830    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-istio-envoy\") pod \"e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22\" (UID: \"5ac35d4e-3dcf-4b95-80c8-90348e8c816a\") " pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"
Jul 23 23:46:51.884897 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:51.884852    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-workload-socket\") pod \"e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22\" (UID: \"5ac35d4e-3dcf-4b95-80c8-90348e8c816a\") " pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"
Jul 23 23:46:51.884897 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:51.884868    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-workload-certs\") pod \"e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22\" (UID: \"5ac35d4e-3dcf-4b95-80c8-90348e8c816a\") " pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"
Jul 23 23:46:51.885051 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:51.884918    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-credential-socket\") pod \"e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22\" (UID: \"5ac35d4e-3dcf-4b95-80c8-90348e8c816a\") " pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"
Jul 23 23:46:51.885051 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:51.884962    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-istio-data\") pod \"e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22\" (UID: \"5ac35d4e-3dcf-4b95-80c8-90348e8c816a\") " pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"
Jul 23 23:46:51.986027 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:51.985988    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-istio-envoy\") pod \"e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22\" (UID: \"5ac35d4e-3dcf-4b95-80c8-90348e8c816a\") " pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"
Jul 23 23:46:51.986027 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:51.986030    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-workload-socket\") pod \"e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22\" (UID: \"5ac35d4e-3dcf-4b95-80c8-90348e8c816a\") " pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"
Jul 23 23:46:51.986311 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:51.986048    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-workload-certs\") pod \"e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22\" (UID: \"5ac35d4e-3dcf-4b95-80c8-90348e8c816a\") " pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"
Jul 23 23:46:51.986311 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:51.986064    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-credential-socket\") pod \"e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22\" (UID: \"5ac35d4e-3dcf-4b95-80c8-90348e8c816a\") " pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"
Jul 23 23:46:51.986311 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:51.986085    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-istio-data\") pod \"e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22\" (UID: \"5ac35d4e-3dcf-4b95-80c8-90348e8c816a\") " pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"
Jul 23 23:46:51.986311 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:51.986166    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-istio-token\") pod \"e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22\" (UID: \"5ac35d4e-3dcf-4b95-80c8-90348e8c816a\") " pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"
Jul 23 23:46:51.986311 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:51.986198    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bngh\" (UniqueName: \"kubernetes.io/projected/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-kube-api-access-2bngh\") pod \"e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22\" (UID: \"5ac35d4e-3dcf-4b95-80c8-90348e8c816a\") " pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"
Jul 23 23:46:51.986311 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:51.986225    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-istio-podinfo\") pod \"e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22\" (UID: \"5ac35d4e-3dcf-4b95-80c8-90348e8c816a\") " pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"
Jul 23 23:46:51.986311 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:51.986290    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-istiod-ca-cert\") pod \"e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22\" (UID: \"5ac35d4e-3dcf-4b95-80c8-90348e8c816a\") " pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"
Jul 23 23:46:51.986671 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:51.986544    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-workload-socket\") pod \"e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22\" (UID: \"5ac35d4e-3dcf-4b95-80c8-90348e8c816a\") " pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"
Jul 23 23:46:51.986671 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:51.986579    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-workload-certs\") pod \"e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22\" (UID: \"5ac35d4e-3dcf-4b95-80c8-90348e8c816a\") " pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"
Jul 23 23:46:51.986779 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:51.986727    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-credential-socket\") pod \"e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22\" (UID: \"5ac35d4e-3dcf-4b95-80c8-90348e8c816a\") " pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"
Jul 23 23:46:51.986975 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:51.986951    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-istio-data\") pod \"e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22\" (UID: \"5ac35d4e-3dcf-4b95-80c8-90348e8c816a\") " pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"
Jul 23 23:46:51.987087 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:51.987071    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-istiod-ca-cert\") pod \"e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22\" (UID: \"5ac35d4e-3dcf-4b95-80c8-90348e8c816a\") " pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"
Jul 23 23:46:51.988422 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:51.988394    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-istio-envoy\") pod \"e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22\" (UID: \"5ac35d4e-3dcf-4b95-80c8-90348e8c816a\") " pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"
Jul 23 23:46:51.988942 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:51.988924    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-istio-podinfo\") pod \"e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22\" (UID: \"5ac35d4e-3dcf-4b95-80c8-90348e8c816a\") " pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"
Jul 23 23:46:51.994254 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:51.994236    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bngh\" (UniqueName: \"kubernetes.io/projected/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-kube-api-access-2bngh\") pod \"e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22\" (UID: \"5ac35d4e-3dcf-4b95-80c8-90348e8c816a\") " pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"
Jul 23 23:46:51.994350 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:51.994324    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-istio-token\") pod \"e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22\" (UID: \"5ac35d4e-3dcf-4b95-80c8-90348e8c816a\") " pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"
Jul 23 23:46:52.061804 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:52.061715    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"
Jul 23 23:46:52.202466 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:52.202383    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"]
Jul 23 23:46:52.205157 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:46:52.205126    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ac35d4e_3dcf_4b95_80c8_90348e8c816a.slice/crio-091b6975f95e134db65c3708580e92c27cab04cb9b96ea9cc43dd247e2b83184 WatchSource:0}: Error finding container 091b6975f95e134db65c3708580e92c27cab04cb9b96ea9cc43dd247e2b83184: Status 404 returned error can't find the container with id 091b6975f95e134db65c3708580e92c27cab04cb9b96ea9cc43dd247e2b83184
Jul 23 23:46:52.207558 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:52.207525    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:46:52.207628 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:52.207611    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:46:52.207687 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:52.207670    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:46:52.644254 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:52.644215    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22" event={"ID":"5ac35d4e-3dcf-4b95-80c8-90348e8c816a","Type":"ContainerStarted","Data":"2c39e25e5716bec948e77e3567d18787da9afa0b1463ed35c9bf03f9cb15a2ab"}
Jul 23 23:46:52.644470 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:52.644277    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22" event={"ID":"5ac35d4e-3dcf-4b95-80c8-90348e8c816a","Type":"ContainerStarted","Data":"091b6975f95e134db65c3708580e92c27cab04cb9b96ea9cc43dd247e2b83184"}
Jul 23 23:46:52.663442 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:52.663393    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22" podStartSLOduration=1.663376857 podStartE2EDuration="1.663376857s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:46:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:46:52.661107843 +0000 UTC m=+2002.030633504" watchObservedRunningTime="2026-07-23 23:46:52.663376857 +0000 UTC m=+2002.032902589"
Jul 23 23:46:53.062224 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:53.062183    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"
Jul 23 23:46:53.063891 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:53.063869    2578 patch_prober.go:28] interesting pod/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Jul 23 23:46:53.063962 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:53.063919    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22" podUID="5ac35d4e-3dcf-4b95-80c8-90348e8c816a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Jul 23 23:46:54.070809 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:54.070770    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"
Jul 23 23:46:54.634096 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:54.634067    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c228ab-86cd756c6-86pgt"
Jul 23 23:46:54.654128 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:54.654068    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"
Jul 23 23:46:54.655739 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:54.655702    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"
Jul 23 23:46:54.677532 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:54.677494    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d7e557-7f9f56d74d-5knxm"]
Jul 23 23:46:54.708948 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:54.708909    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d7e557-7f9f56d74d-5knxm"]
Jul 23 23:46:54.709163 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:54.709040    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d7e557-7f9f56d74d-5knxm"
Jul 23 23:46:54.712309 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:54.711309    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmgmq\" (UniqueName: \"kubernetes.io/projected/3aee1e72-ed31-45f1-bb62-39a615ad5e7d-kube-api-access-xmgmq\") pod \"maas-api-e2e-inf-b-d7e557-7f9f56d74d-5knxm\" (UID: \"3aee1e72-ed31-45f1-bb62-39a615ad5e7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d7e557-7f9f56d74d-5knxm"
Jul 23 23:46:54.712309 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:54.711362    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3aee1e72-ed31-45f1-bb62-39a615ad5e7d-maas-api-tls\") pod \"maas-api-e2e-inf-b-d7e557-7f9f56d74d-5knxm\" (UID: \"3aee1e72-ed31-45f1-bb62-39a615ad5e7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d7e557-7f9f56d74d-5knxm"
Jul 23 23:46:54.712309 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:54.711465    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3aee1e72-ed31-45f1-bb62-39a615ad5e7d-openshift-service-ca\") pod \"maas-api-e2e-inf-b-d7e557-7f9f56d74d-5knxm\" (UID: \"3aee1e72-ed31-45f1-bb62-39a615ad5e7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d7e557-7f9f56d74d-5knxm"
Jul 23 23:46:54.713767 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:54.713745    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-d7e557-serving-cert\""
Jul 23 23:46:54.812867 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:54.812827    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmgmq\" (UniqueName: \"kubernetes.io/projected/3aee1e72-ed31-45f1-bb62-39a615ad5e7d-kube-api-access-xmgmq\") pod \"maas-api-e2e-inf-b-d7e557-7f9f56d74d-5knxm\" (UID: \"3aee1e72-ed31-45f1-bb62-39a615ad5e7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d7e557-7f9f56d74d-5knxm"
Jul 23 23:46:54.812867 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:54.812872    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3aee1e72-ed31-45f1-bb62-39a615ad5e7d-maas-api-tls\") pod \"maas-api-e2e-inf-b-d7e557-7f9f56d74d-5knxm\" (UID: \"3aee1e72-ed31-45f1-bb62-39a615ad5e7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d7e557-7f9f56d74d-5knxm"
Jul 23 23:46:54.813133 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:54.812913    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3aee1e72-ed31-45f1-bb62-39a615ad5e7d-openshift-service-ca\") pod \"maas-api-e2e-inf-b-d7e557-7f9f56d74d-5knxm\" (UID: \"3aee1e72-ed31-45f1-bb62-39a615ad5e7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d7e557-7f9f56d74d-5knxm"
Jul 23 23:46:54.813537 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:54.813514    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3aee1e72-ed31-45f1-bb62-39a615ad5e7d-openshift-service-ca\") pod \"maas-api-e2e-inf-b-d7e557-7f9f56d74d-5knxm\" (UID: \"3aee1e72-ed31-45f1-bb62-39a615ad5e7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d7e557-7f9f56d74d-5knxm"
Jul 23 23:46:54.815767 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:54.815741    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3aee1e72-ed31-45f1-bb62-39a615ad5e7d-maas-api-tls\") pod \"maas-api-e2e-inf-b-d7e557-7f9f56d74d-5knxm\" (UID: \"3aee1e72-ed31-45f1-bb62-39a615ad5e7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d7e557-7f9f56d74d-5knxm"
Jul 23 23:46:54.820768 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:54.820747    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmgmq\" (UniqueName: \"kubernetes.io/projected/3aee1e72-ed31-45f1-bb62-39a615ad5e7d-kube-api-access-xmgmq\") pod \"maas-api-e2e-inf-b-d7e557-7f9f56d74d-5knxm\" (UID: \"3aee1e72-ed31-45f1-bb62-39a615ad5e7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d7e557-7f9f56d74d-5knxm"
Jul 23 23:46:55.024663 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:55.024611    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d7e557-7f9f56d74d-5knxm"
Jul 23 23:46:55.199717 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:55.199350    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d7e557-7f9f56d74d-5knxm"]
Jul 23 23:46:55.200695 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:46:55.200643    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3aee1e72_ed31_45f1_bb62_39a615ad5e7d.slice/crio-8875ab488e4c719ed92f428293b1fa0f847312a4c63a1b6dc301567133222965 WatchSource:0}: Error finding container 8875ab488e4c719ed92f428293b1fa0f847312a4c63a1b6dc301567133222965: Status 404 returned error can't find the container with id 8875ab488e4c719ed92f428293b1fa0f847312a4c63a1b6dc301567133222965
Jul 23 23:46:55.660731 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:55.660688    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d7e557-7f9f56d74d-5knxm" event={"ID":"3aee1e72-ed31-45f1-bb62-39a615ad5e7d","Type":"ContainerStarted","Data":"7fe7fed4c74aa695a5e5fd7f243a63b945fc9b2ce2e2eac15a3ce44309ce415c"}
Jul 23 23:46:55.660934 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:55.660767    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d7e557-7f9f56d74d-5knxm"
Jul 23 23:46:55.660934 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:55.660791    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d7e557-7f9f56d74d-5knxm" event={"ID":"3aee1e72-ed31-45f1-bb62-39a615ad5e7d","Type":"ContainerStarted","Data":"8875ab488e4c719ed92f428293b1fa0f847312a4c63a1b6dc301567133222965"}
Jul 23 23:46:59.682613 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:59.682578    2578 generic.go:356] "Generic (PLEG): container finished" podID="5492bdc3-62e0-43c9-b6e9-cb334db3a3ea" containerID="16fbb97be2754d42445a5633a5bd5c1af291a9c9ebf0717c56799f7376c8b480" exitCode=0
Jul 23 23:46:59.683169 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:59.682688    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw" event={"ID":"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea","Type":"ContainerDied","Data":"16fbb97be2754d42445a5633a5bd5c1af291a9c9ebf0717c56799f7376c8b480"}
Jul 23 23:46:59.683303 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:59.683284    2578 scope.go:117] "RemoveContainer" containerID="16fbb97be2754d42445a5633a5bd5c1af291a9c9ebf0717c56799f7376c8b480"
Jul 23 23:46:59.684477 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:59.684445    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:46:59.684550 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:59.684532    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:46:59.684592 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:59.684575    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:46:59.703858 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:46:59.703805    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d7e557-7f9f56d74d-5knxm" podStartSLOduration=5.403631178 podStartE2EDuration="5.703787673s" totalImagesPullingTime="300.156495ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:46:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:46:55.679826921 +0000 UTC m=+2005.049352580" watchObservedRunningTime="2026-07-23 23:46:59.703787673 +0000 UTC m=+2009.073313332"
Jul 23 23:47:00.607434 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:47:00.607396    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"
Jul 23 23:47:00.693141 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:47:00.693077    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw" event={"ID":"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea","Type":"ContainerStarted","Data":"8bd14058a0584694e058904973437f3c3b11ae53031a3bc6b0f30f2948284d68"}
Jul 23 23:47:01.004504 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:47:01.004472    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"
Jul 23 23:47:01.005342 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:47:01.005313    2578 patch_prober.go:28] interesting pod/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 23 23:47:01.005475 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:47:01.005376    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw" podUID="5492bdc3-62e0-43c9-b6e9-cb334db3a3ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 23 23:47:01.687990 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:47:01.687957    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d7e557-7f9f56d74d-5knxm"
Jul 23 23:47:02.005191 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:47:02.005086    2578 patch_prober.go:28] interesting pod/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 23 23:47:02.005191 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:47:02.005169    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw" podUID="5492bdc3-62e0-43c9-b6e9-cb334db3a3ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 23 23:47:03.005387 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:47:03.005337    2578 patch_prober.go:28] interesting pod/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 23 23:47:03.005941 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:47:03.005408    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw" podUID="5492bdc3-62e0-43c9-b6e9-cb334db3a3ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 23 23:47:04.005423 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:47:04.004924    2578 patch_prober.go:28] interesting pod/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 23 23:47:04.005423 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:47:04.004989    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw" podUID="5492bdc3-62e0-43c9-b6e9-cb334db3a3ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 23 23:47:05.008408 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:47:05.008370    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"
Jul 23 23:47:05.008915 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:47:05.008647    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"
Jul 23 23:47:05.009426 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:47:05.009403    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"
Jul 23 23:47:08.760094 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:47:08.760059    2578 generic.go:356] "Generic (PLEG): container finished" podID="5ac35d4e-3dcf-4b95-80c8-90348e8c816a" containerID="2c39e25e5716bec948e77e3567d18787da9afa0b1463ed35c9bf03f9cb15a2ab" exitCode=0
Jul 23 23:47:08.760560 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:47:08.760127    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22" event={"ID":"5ac35d4e-3dcf-4b95-80c8-90348e8c816a","Type":"ContainerDied","Data":"2c39e25e5716bec948e77e3567d18787da9afa0b1463ed35c9bf03f9cb15a2ab"}
Jul 23 23:47:08.760560 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:47:08.760523    2578 scope.go:117] "RemoveContainer" containerID="2c39e25e5716bec948e77e3567d18787da9afa0b1463ed35c9bf03f9cb15a2ab"
Jul 23 23:47:08.761577 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:47:08.761547    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:47:08.761675 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:47:08.761639    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:47:08.761720 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:47:08.761708    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:47:09.654604 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:47:09.654574    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"
Jul 23 23:47:09.767142 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:47:09.767102    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22" event={"ID":"5ac35d4e-3dcf-4b95-80c8-90348e8c816a","Type":"ContainerStarted","Data":"64d3a77107632083afb189dd620260eba8e0e4414424a121fccb1f205751dc64"}
Jul 23 23:47:10.062853 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:47:10.062812    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"
Jul 23 23:47:10.064094 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:47:10.064062    2578 patch_prober.go:28] interesting pod/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Jul 23 23:47:10.064224 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:47:10.064125    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22" podUID="5ac35d4e-3dcf-4b95-80c8-90348e8c816a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Jul 23 23:47:11.063038 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:47:11.062996    2578 patch_prober.go:28] interesting pod/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Jul 23 23:47:11.063451 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:47:11.063054    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22" podUID="5ac35d4e-3dcf-4b95-80c8-90348e8c816a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Jul 23 23:47:12.062536 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:47:12.062497    2578 patch_prober.go:28] interesting pod/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Jul 23 23:47:12.062760 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:47:12.062556    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22" podUID="5ac35d4e-3dcf-4b95-80c8-90348e8c816a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Jul 23 23:47:13.062880 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:47:13.062840    2578 patch_prober.go:28] interesting pod/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Jul 23 23:47:13.063364 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:47:13.062905    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22" podUID="5ac35d4e-3dcf-4b95-80c8-90348e8c816a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Jul 23 23:47:14.067897 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:47:14.067860    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"
Jul 23 23:47:14.068315 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:47:14.068153    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"
Jul 23 23:47:14.069111 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:47:14.069087    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"
Jul 23 23:48:19.879232 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:19.879197    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c228ab-86cd756c6-86pgt"]
Jul 23 23:48:19.879773 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:19.879501    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c228ab-86cd756c6-86pgt" podUID="290afd93-a50f-441b-a5f0-25170ec4f048" containerName="maas-api" containerID="cri-o://52ac49d456d63aa677c477d1928fad571426ec6bfdef446e733a8b61563de24f" gracePeriod=30
Jul 23 23:48:20.122056 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:20.122026    2578 generic.go:356] "Generic (PLEG): container finished" podID="290afd93-a50f-441b-a5f0-25170ec4f048" containerID="52ac49d456d63aa677c477d1928fad571426ec6bfdef446e733a8b61563de24f" exitCode=0
Jul 23 23:48:20.122176 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:20.122074    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c228ab-86cd756c6-86pgt" event={"ID":"290afd93-a50f-441b-a5f0-25170ec4f048","Type":"ContainerDied","Data":"52ac49d456d63aa677c477d1928fad571426ec6bfdef446e733a8b61563de24f"}
Jul 23 23:48:20.141193 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:20.141135    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c228ab-86cd756c6-86pgt"
Jul 23 23:48:20.175004 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:20.174978    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q2lxb\" (UniqueName: \"kubernetes.io/projected/290afd93-a50f-441b-a5f0-25170ec4f048-kube-api-access-q2lxb\") pod \"290afd93-a50f-441b-a5f0-25170ec4f048\" (UID: \"290afd93-a50f-441b-a5f0-25170ec4f048\") "
Jul 23 23:48:20.175178 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:20.175084    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/290afd93-a50f-441b-a5f0-25170ec4f048-maas-api-tls\") pod \"290afd93-a50f-441b-a5f0-25170ec4f048\" (UID: \"290afd93-a50f-441b-a5f0-25170ec4f048\") "
Jul 23 23:48:20.175178 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:20.175104    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/290afd93-a50f-441b-a5f0-25170ec4f048-openshift-service-ca\") pod \"290afd93-a50f-441b-a5f0-25170ec4f048\" (UID: \"290afd93-a50f-441b-a5f0-25170ec4f048\") "
Jul 23 23:48:20.175482 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:20.175456    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/290afd93-a50f-441b-a5f0-25170ec4f048-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "290afd93-a50f-441b-a5f0-25170ec4f048" (UID: "290afd93-a50f-441b-a5f0-25170ec4f048"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:48:20.177449 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:20.177417    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/290afd93-a50f-441b-a5f0-25170ec4f048-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "290afd93-a50f-441b-a5f0-25170ec4f048" (UID: "290afd93-a50f-441b-a5f0-25170ec4f048"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 23:48:20.177570 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:20.177479    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/290afd93-a50f-441b-a5f0-25170ec4f048-kube-api-access-q2lxb" (OuterVolumeSpecName: "kube-api-access-q2lxb") pod "290afd93-a50f-441b-a5f0-25170ec4f048" (UID: "290afd93-a50f-441b-a5f0-25170ec4f048"). InnerVolumeSpecName "kube-api-access-q2lxb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:48:20.276485 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:20.276449    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2lxb\" (UniqueName: \"kubernetes.io/projected/290afd93-a50f-441b-a5f0-25170ec4f048-kube-api-access-q2lxb\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:48:20.276485 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:20.276476    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/290afd93-a50f-441b-a5f0-25170ec4f048-maas-api-tls\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:48:20.276485 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:20.276485    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/290afd93-a50f-441b-a5f0-25170ec4f048-openshift-service-ca\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:48:21.128301 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:21.128268    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c228ab-86cd756c6-86pgt"
Jul 23 23:48:21.128301 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:21.128281    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c228ab-86cd756c6-86pgt" event={"ID":"290afd93-a50f-441b-a5f0-25170ec4f048","Type":"ContainerDied","Data":"13c0f68c20664b77eedab1b21948c8fd501d1e3d4701c2a5bbe84591202ff4db"}
Jul 23 23:48:21.128839 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:21.128354    2578 scope.go:117] "RemoveContainer" containerID="52ac49d456d63aa677c477d1928fad571426ec6bfdef446e733a8b61563de24f"
Jul 23 23:48:21.152423 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:21.152396    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c228ab-86cd756c6-86pgt"]
Jul 23 23:48:21.155981 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:21.155957    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c228ab-86cd756c6-86pgt"]
Jul 23 23:48:21.236830 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:21.236795    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="290afd93-a50f-441b-a5f0-25170ec4f048" path="/var/lib/kubelet/pods/290afd93-a50f-441b-a5f0-25170ec4f048/volumes"
Jul 23 23:48:31.117876 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:31.117842    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"]
Jul 23 23:48:31.118335 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:31.118065    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw" podUID="5492bdc3-62e0-43c9-b6e9-cb334db3a3ea" containerName="istio-proxy" containerID="cri-o://8bd14058a0584694e058904973437f3c3b11ae53031a3bc6b0f30f2948284d68" gracePeriod=30
Jul 23 23:48:35.010951 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:35.010913    2578 patch_prober.go:28] interesting pod/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 23 23:48:35.011316 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:35.010983    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw" podUID="5492bdc3-62e0-43c9-b6e9-cb334db3a3ea" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 23 23:48:36.378914 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:36.378850    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"
Jul 23 23:48:36.529721 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:36.529692    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-istio-podinfo\") pod \"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea\" (UID: \"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea\") "
Jul 23 23:48:36.529874 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:36.529753    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-credential-socket\") pod \"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea\" (UID: \"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea\") "
Jul 23 23:48:36.529874 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:36.529780    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-istio-data\") pod \"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea\" (UID: \"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea\") "
Jul 23 23:48:36.529874 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:36.529803    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-istio-envoy\") pod \"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea\" (UID: \"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea\") "
Jul 23 23:48:36.529874 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:36.529834    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-istio-token\") pod \"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea\" (UID: \"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea\") "
Jul 23 23:48:36.529874 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:36.529857    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-workload-socket\") pod \"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea\" (UID: \"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea\") "
Jul 23 23:48:36.530208 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:36.529882    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-workload-certs\") pod \"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea\" (UID: \"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea\") "
Jul 23 23:48:36.530208 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:36.529916    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-istiod-ca-cert\") pod \"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea\" (UID: \"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea\") "
Jul 23 23:48:36.530208 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:36.529949    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rbzp4\" (UniqueName: \"kubernetes.io/projected/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-kube-api-access-rbzp4\") pod \"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea\" (UID: \"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea\") "
Jul 23 23:48:36.530208 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:36.530018    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5492bdc3-62e0-43c9-b6e9-cb334db3a3ea" (UID: "5492bdc3-62e0-43c9-b6e9-cb334db3a3ea"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:48:36.530408 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:36.530212    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5492bdc3-62e0-43c9-b6e9-cb334db3a3ea" (UID: "5492bdc3-62e0-43c9-b6e9-cb334db3a3ea"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:48:36.530408 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:36.530321    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5492bdc3-62e0-43c9-b6e9-cb334db3a3ea" (UID: "5492bdc3-62e0-43c9-b6e9-cb334db3a3ea"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:48:36.530408 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:36.530339    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5492bdc3-62e0-43c9-b6e9-cb334db3a3ea" (UID: "5492bdc3-62e0-43c9-b6e9-cb334db3a3ea"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:48:36.530408 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:36.530350    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-workload-certs\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:48:36.530408 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:36.530370    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-credential-socket\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:48:36.531013 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:36.530987    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-istio-data" (OuterVolumeSpecName: "istio-data") pod "5492bdc3-62e0-43c9-b6e9-cb334db3a3ea" (UID: "5492bdc3-62e0-43c9-b6e9-cb334db3a3ea"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:48:36.532173 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:36.532151    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5492bdc3-62e0-43c9-b6e9-cb334db3a3ea" (UID: "5492bdc3-62e0-43c9-b6e9-cb334db3a3ea"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 23:48:36.532173 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:36.532161    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5492bdc3-62e0-43c9-b6e9-cb334db3a3ea" (UID: "5492bdc3-62e0-43c9-b6e9-cb334db3a3ea"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:48:36.532703 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:36.532685    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-kube-api-access-rbzp4" (OuterVolumeSpecName: "kube-api-access-rbzp4") pod "5492bdc3-62e0-43c9-b6e9-cb334db3a3ea" (UID: "5492bdc3-62e0-43c9-b6e9-cb334db3a3ea"). InnerVolumeSpecName "kube-api-access-rbzp4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:48:36.532780 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:36.532748    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-istio-token" (OuterVolumeSpecName: "istio-token") pod "5492bdc3-62e0-43c9-b6e9-cb334db3a3ea" (UID: "5492bdc3-62e0-43c9-b6e9-cb334db3a3ea"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:48:36.631521 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:36.631420    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-istio-podinfo\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:48:36.631521 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:36.631464    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-istio-data\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:48:36.631521 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:36.631474    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-istio-envoy\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:48:36.631521 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:36.631482    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-istio-token\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:48:36.631521 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:36.631490    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-workload-socket\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:48:36.631521 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:36.631498    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-istiod-ca-cert\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:48:36.631521 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:36.631506    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbzp4\" (UniqueName: \"kubernetes.io/projected/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea-kube-api-access-rbzp4\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:48:37.206027 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:37.205992    2578 generic.go:356] "Generic (PLEG): container finished" podID="5492bdc3-62e0-43c9-b6e9-cb334db3a3ea" containerID="8bd14058a0584694e058904973437f3c3b11ae53031a3bc6b0f30f2948284d68" exitCode=0
Jul 23 23:48:37.206228 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:37.206041    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"
Jul 23 23:48:37.206228 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:37.206084    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw" event={"ID":"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea","Type":"ContainerDied","Data":"8bd14058a0584694e058904973437f3c3b11ae53031a3bc6b0f30f2948284d68"}
Jul 23 23:48:37.206228 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:37.206147    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw" event={"ID":"5492bdc3-62e0-43c9-b6e9-cb334db3a3ea","Type":"ContainerDied","Data":"5d3bcb9a34cde387cb53ea69e43368d5f6dad0e650196fd367a4ab2dec4766a5"}
Jul 23 23:48:37.206228 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:37.206176    2578 scope.go:117] "RemoveContainer" containerID="8bd14058a0584694e058904973437f3c3b11ae53031a3bc6b0f30f2948284d68"
Jul 23 23:48:37.215459 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:37.215444    2578 scope.go:117] "RemoveContainer" containerID="16fbb97be2754d42445a5633a5bd5c1af291a9c9ebf0717c56799f7376c8b480"
Jul 23 23:48:37.223207 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:37.223189    2578 scope.go:117] "RemoveContainer" containerID="8bd14058a0584694e058904973437f3c3b11ae53031a3bc6b0f30f2948284d68"
Jul 23 23:48:37.223451 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:48:37.223432    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bd14058a0584694e058904973437f3c3b11ae53031a3bc6b0f30f2948284d68\": container with ID starting with 8bd14058a0584694e058904973437f3c3b11ae53031a3bc6b0f30f2948284d68 not found: ID does not exist" containerID="8bd14058a0584694e058904973437f3c3b11ae53031a3bc6b0f30f2948284d68"
Jul 23 23:48:37.223552 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:37.223458    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bd14058a0584694e058904973437f3c3b11ae53031a3bc6b0f30f2948284d68"} err="failed to get container status \"8bd14058a0584694e058904973437f3c3b11ae53031a3bc6b0f30f2948284d68\": rpc error: code = NotFound desc = could not find container \"8bd14058a0584694e058904973437f3c3b11ae53031a3bc6b0f30f2948284d68\": container with ID starting with 8bd14058a0584694e058904973437f3c3b11ae53031a3bc6b0f30f2948284d68 not found: ID does not exist"
Jul 23 23:48:37.223552 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:37.223475    2578 scope.go:117] "RemoveContainer" containerID="16fbb97be2754d42445a5633a5bd5c1af291a9c9ebf0717c56799f7376c8b480"
Jul 23 23:48:37.223732 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:48:37.223714    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16fbb97be2754d42445a5633a5bd5c1af291a9c9ebf0717c56799f7376c8b480\": container with ID starting with 16fbb97be2754d42445a5633a5bd5c1af291a9c9ebf0717c56799f7376c8b480 not found: ID does not exist" containerID="16fbb97be2754d42445a5633a5bd5c1af291a9c9ebf0717c56799f7376c8b480"
Jul 23 23:48:37.223786 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:37.223739    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16fbb97be2754d42445a5633a5bd5c1af291a9c9ebf0717c56799f7376c8b480"} err="failed to get container status \"16fbb97be2754d42445a5633a5bd5c1af291a9c9ebf0717c56799f7376c8b480\": rpc error: code = NotFound desc = could not find container \"16fbb97be2754d42445a5633a5bd5c1af291a9c9ebf0717c56799f7376c8b480\": container with ID starting with 16fbb97be2754d42445a5633a5bd5c1af291a9c9ebf0717c56799f7376c8b480 not found: ID does not exist"
Jul 23 23:48:37.227764 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:37.227743    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"]
Jul 23 23:48:37.236100 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:37.236081    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-c228ab-openshift-default-5478b95bcd-drpvw"]
Jul 23 23:48:39.236343 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:39.236311    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5492bdc3-62e0-43c9-b6e9-cb334db3a3ea" path="/var/lib/kubelet/pods/5492bdc3-62e0-43c9-b6e9-cb334db3a3ea/volumes"
Jul 23 23:48:41.970273 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:41.970238    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d7e557-7f9f56d74d-5knxm"]
Jul 23 23:48:41.970709 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:41.970471    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d7e557-7f9f56d74d-5knxm" podUID="3aee1e72-ed31-45f1-bb62-39a615ad5e7d" containerName="maas-api" containerID="cri-o://7fe7fed4c74aa695a5e5fd7f243a63b945fc9b2ce2e2eac15a3ce44309ce415c" gracePeriod=30
Jul 23 23:48:42.224774 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:42.224711    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d7e557-7f9f56d74d-5knxm"
Jul 23 23:48:42.229768 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:42.229741    2578 generic.go:356] "Generic (PLEG): container finished" podID="3aee1e72-ed31-45f1-bb62-39a615ad5e7d" containerID="7fe7fed4c74aa695a5e5fd7f243a63b945fc9b2ce2e2eac15a3ce44309ce415c" exitCode=0
Jul 23 23:48:42.229891 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:42.229788    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d7e557-7f9f56d74d-5knxm"
Jul 23 23:48:42.229891 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:42.229826    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d7e557-7f9f56d74d-5knxm" event={"ID":"3aee1e72-ed31-45f1-bb62-39a615ad5e7d","Type":"ContainerDied","Data":"7fe7fed4c74aa695a5e5fd7f243a63b945fc9b2ce2e2eac15a3ce44309ce415c"}
Jul 23 23:48:42.229891 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:42.229879    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d7e557-7f9f56d74d-5knxm" event={"ID":"3aee1e72-ed31-45f1-bb62-39a615ad5e7d","Type":"ContainerDied","Data":"8875ab488e4c719ed92f428293b1fa0f847312a4c63a1b6dc301567133222965"}
Jul 23 23:48:42.230022 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:42.229909    2578 scope.go:117] "RemoveContainer" containerID="7fe7fed4c74aa695a5e5fd7f243a63b945fc9b2ce2e2eac15a3ce44309ce415c"
Jul 23 23:48:42.239597 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:42.239577    2578 scope.go:117] "RemoveContainer" containerID="7fe7fed4c74aa695a5e5fd7f243a63b945fc9b2ce2e2eac15a3ce44309ce415c"
Jul 23 23:48:42.239892 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:48:42.239871    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fe7fed4c74aa695a5e5fd7f243a63b945fc9b2ce2e2eac15a3ce44309ce415c\": container with ID starting with 7fe7fed4c74aa695a5e5fd7f243a63b945fc9b2ce2e2eac15a3ce44309ce415c not found: ID does not exist" containerID="7fe7fed4c74aa695a5e5fd7f243a63b945fc9b2ce2e2eac15a3ce44309ce415c"
Jul 23 23:48:42.239971 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:42.239904    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fe7fed4c74aa695a5e5fd7f243a63b945fc9b2ce2e2eac15a3ce44309ce415c"} err="failed to get container status \"7fe7fed4c74aa695a5e5fd7f243a63b945fc9b2ce2e2eac15a3ce44309ce415c\": rpc error: code = NotFound desc = could not find container \"7fe7fed4c74aa695a5e5fd7f243a63b945fc9b2ce2e2eac15a3ce44309ce415c\": container with ID starting with 7fe7fed4c74aa695a5e5fd7f243a63b945fc9b2ce2e2eac15a3ce44309ce415c not found: ID does not exist"
Jul 23 23:48:42.390886 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:42.390847    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3aee1e72-ed31-45f1-bb62-39a615ad5e7d-maas-api-tls\") pod \"3aee1e72-ed31-45f1-bb62-39a615ad5e7d\" (UID: \"3aee1e72-ed31-45f1-bb62-39a615ad5e7d\") "
Jul 23 23:48:42.391089 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:42.390941    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xmgmq\" (UniqueName: \"kubernetes.io/projected/3aee1e72-ed31-45f1-bb62-39a615ad5e7d-kube-api-access-xmgmq\") pod \"3aee1e72-ed31-45f1-bb62-39a615ad5e7d\" (UID: \"3aee1e72-ed31-45f1-bb62-39a615ad5e7d\") "
Jul 23 23:48:42.391089 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:42.391054    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3aee1e72-ed31-45f1-bb62-39a615ad5e7d-openshift-service-ca\") pod \"3aee1e72-ed31-45f1-bb62-39a615ad5e7d\" (UID: \"3aee1e72-ed31-45f1-bb62-39a615ad5e7d\") "
Jul 23 23:48:42.391379 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:42.391352    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3aee1e72-ed31-45f1-bb62-39a615ad5e7d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3aee1e72-ed31-45f1-bb62-39a615ad5e7d" (UID: "3aee1e72-ed31-45f1-bb62-39a615ad5e7d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:48:42.393201 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:42.393174    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3aee1e72-ed31-45f1-bb62-39a615ad5e7d-kube-api-access-xmgmq" (OuterVolumeSpecName: "kube-api-access-xmgmq") pod "3aee1e72-ed31-45f1-bb62-39a615ad5e7d" (UID: "3aee1e72-ed31-45f1-bb62-39a615ad5e7d"). InnerVolumeSpecName "kube-api-access-xmgmq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:48:42.393316 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:42.393249    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3aee1e72-ed31-45f1-bb62-39a615ad5e7d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3aee1e72-ed31-45f1-bb62-39a615ad5e7d" (UID: "3aee1e72-ed31-45f1-bb62-39a615ad5e7d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 23:48:42.492774 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:42.492644    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmgmq\" (UniqueName: \"kubernetes.io/projected/3aee1e72-ed31-45f1-bb62-39a615ad5e7d-kube-api-access-xmgmq\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:48:42.492774 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:42.492713    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3aee1e72-ed31-45f1-bb62-39a615ad5e7d-openshift-service-ca\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:48:42.492774 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:42.492724    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3aee1e72-ed31-45f1-bb62-39a615ad5e7d-maas-api-tls\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:48:42.551736 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:42.551691    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d7e557-7f9f56d74d-5knxm"]
Jul 23 23:48:42.557458 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:42.557428    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d7e557-7f9f56d74d-5knxm"]
Jul 23 23:48:43.238801 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:43.238768    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3aee1e72-ed31-45f1-bb62-39a615ad5e7d" path="/var/lib/kubelet/pods/3aee1e72-ed31-45f1-bb62-39a615ad5e7d/volumes"
Jul 23 23:48:52.788524 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:52.788489    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"]
Jul 23 23:48:52.789141 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:52.788722    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22" podUID="5ac35d4e-3dcf-4b95-80c8-90348e8c816a" containerName="istio-proxy" containerID="cri-o://64d3a77107632083afb189dd620260eba8e0e4414424a121fccb1f205751dc64" gracePeriod=30
Jul 23 23:48:53.685896 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:53.685855    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v"]
Jul 23 23:48:53.686729 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:53.686703    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5492bdc3-62e0-43c9-b6e9-cb334db3a3ea" containerName="istio-proxy"
Jul 23 23:48:53.686729 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:53.686729    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5492bdc3-62e0-43c9-b6e9-cb334db3a3ea" containerName="istio-proxy"
Jul 23 23:48:53.686912 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:53.686742    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="290afd93-a50f-441b-a5f0-25170ec4f048" containerName="maas-api"
Jul 23 23:48:53.686912 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:53.686757    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="290afd93-a50f-441b-a5f0-25170ec4f048" containerName="maas-api"
Jul 23 23:48:53.686912 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:53.686789    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3aee1e72-ed31-45f1-bb62-39a615ad5e7d" containerName="maas-api"
Jul 23 23:48:53.686912 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:53.686798    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3aee1e72-ed31-45f1-bb62-39a615ad5e7d" containerName="maas-api"
Jul 23 23:48:53.687099 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:53.686961    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="290afd93-a50f-441b-a5f0-25170ec4f048" containerName="maas-api"
Jul 23 23:48:53.687099 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:53.686972    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5492bdc3-62e0-43c9-b6e9-cb334db3a3ea" containerName="istio-proxy"
Jul 23 23:48:53.687099 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:53.686984    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3aee1e72-ed31-45f1-bb62-39a615ad5e7d" containerName="maas-api"
Jul 23 23:48:53.687099 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:53.687000    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5492bdc3-62e0-43c9-b6e9-cb334db3a3ea" containerName="istio-proxy"
Jul 23 23:48:53.692284 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:53.692255    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v"
Jul 23 23:48:53.695110 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:53.695085    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-ce89b1-openshift-default-dockercfg-l72dc\""
Jul 23 23:48:53.699478 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:53.699452    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v"]
Jul 23 23:48:53.792278 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:53.792240    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fa76844-7596-4283-9737-ee24b56f3202-istio-token\") pod \"e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v\" (UID: \"9fa76844-7596-4283-9737-ee24b56f3202\") " pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v"
Jul 23 23:48:53.792278 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:53.792278    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cfb5\" (UniqueName: \"kubernetes.io/projected/9fa76844-7596-4283-9737-ee24b56f3202-kube-api-access-8cfb5\") pod \"e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v\" (UID: \"9fa76844-7596-4283-9737-ee24b56f3202\") " pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v"
Jul 23 23:48:53.792719 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:53.792303    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fa76844-7596-4283-9737-ee24b56f3202-istiod-ca-cert\") pod \"e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v\" (UID: \"9fa76844-7596-4283-9737-ee24b56f3202\") " pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v"
Jul 23 23:48:53.792719 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:53.792372    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fa76844-7596-4283-9737-ee24b56f3202-istio-podinfo\") pod \"e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v\" (UID: \"9fa76844-7596-4283-9737-ee24b56f3202\") " pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v"
Jul 23 23:48:53.792719 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:53.792416    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fa76844-7596-4283-9737-ee24b56f3202-workload-certs\") pod \"e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v\" (UID: \"9fa76844-7596-4283-9737-ee24b56f3202\") " pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v"
Jul 23 23:48:53.792719 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:53.792436    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fa76844-7596-4283-9737-ee24b56f3202-credential-socket\") pod \"e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v\" (UID: \"9fa76844-7596-4283-9737-ee24b56f3202\") " pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v"
Jul 23 23:48:53.792719 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:53.792462    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fa76844-7596-4283-9737-ee24b56f3202-istio-envoy\") pod \"e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v\" (UID: \"9fa76844-7596-4283-9737-ee24b56f3202\") " pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v"
Jul 23 23:48:53.792719 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:53.792520    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fa76844-7596-4283-9737-ee24b56f3202-workload-socket\") pod \"e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v\" (UID: \"9fa76844-7596-4283-9737-ee24b56f3202\") " pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v"
Jul 23 23:48:53.792719 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:53.792560    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fa76844-7596-4283-9737-ee24b56f3202-istio-data\") pod \"e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v\" (UID: \"9fa76844-7596-4283-9737-ee24b56f3202\") " pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v"
Jul 23 23:48:53.893814 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:53.893773    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fa76844-7596-4283-9737-ee24b56f3202-istio-data\") pod \"e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v\" (UID: \"9fa76844-7596-4283-9737-ee24b56f3202\") " pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v"
Jul 23 23:48:53.893997 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:53.893837    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fa76844-7596-4283-9737-ee24b56f3202-istio-token\") pod \"e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v\" (UID: \"9fa76844-7596-4283-9737-ee24b56f3202\") " pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v"
Jul 23 23:48:53.893997 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:53.893964    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cfb5\" (UniqueName: \"kubernetes.io/projected/9fa76844-7596-4283-9737-ee24b56f3202-kube-api-access-8cfb5\") pod \"e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v\" (UID: \"9fa76844-7596-4283-9737-ee24b56f3202\") " pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v"
Jul 23 23:48:53.894078 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:53.894005    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fa76844-7596-4283-9737-ee24b56f3202-istiod-ca-cert\") pod \"e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v\" (UID: \"9fa76844-7596-4283-9737-ee24b56f3202\") " pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v"
Jul 23 23:48:53.894078 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:53.894037    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fa76844-7596-4283-9737-ee24b56f3202-istio-podinfo\") pod \"e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v\" (UID: \"9fa76844-7596-4283-9737-ee24b56f3202\") " pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v"
Jul 23 23:48:53.894159 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:53.894080    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fa76844-7596-4283-9737-ee24b56f3202-workload-certs\") pod \"e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v\" (UID: \"9fa76844-7596-4283-9737-ee24b56f3202\") " pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v"
Jul 23 23:48:53.894159 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:53.894097    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fa76844-7596-4283-9737-ee24b56f3202-credential-socket\") pod \"e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v\" (UID: \"9fa76844-7596-4283-9737-ee24b56f3202\") " pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v"
Jul 23 23:48:53.894159 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:53.894128    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fa76844-7596-4283-9737-ee24b56f3202-istio-envoy\") pod \"e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v\" (UID: \"9fa76844-7596-4283-9737-ee24b56f3202\") " pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v"
Jul 23 23:48:53.894315 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:53.894258    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fa76844-7596-4283-9737-ee24b56f3202-workload-socket\") pod \"e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v\" (UID: \"9fa76844-7596-4283-9737-ee24b56f3202\") " pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v"
Jul 23 23:48:53.894429 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:53.894401    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fa76844-7596-4283-9737-ee24b56f3202-credential-socket\") pod \"e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v\" (UID: \"9fa76844-7596-4283-9737-ee24b56f3202\") " pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v"
Jul 23 23:48:53.894429 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:53.894415    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fa76844-7596-4283-9737-ee24b56f3202-istio-data\") pod \"e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v\" (UID: \"9fa76844-7596-4283-9737-ee24b56f3202\") " pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v"
Jul 23 23:48:53.894587 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:53.894480    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fa76844-7596-4283-9737-ee24b56f3202-workload-certs\") pod \"e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v\" (UID: \"9fa76844-7596-4283-9737-ee24b56f3202\") " pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v"
Jul 23 23:48:53.894633 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:53.894604    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fa76844-7596-4283-9737-ee24b56f3202-workload-socket\") pod \"e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v\" (UID: \"9fa76844-7596-4283-9737-ee24b56f3202\") " pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v"
Jul 23 23:48:53.894736 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:53.894720    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fa76844-7596-4283-9737-ee24b56f3202-istiod-ca-cert\") pod \"e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v\" (UID: \"9fa76844-7596-4283-9737-ee24b56f3202\") " pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v"
Jul 23 23:48:53.896540 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:53.896519    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fa76844-7596-4283-9737-ee24b56f3202-istio-envoy\") pod \"e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v\" (UID: \"9fa76844-7596-4283-9737-ee24b56f3202\") " pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v"
Jul 23 23:48:53.896725 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:53.896706    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fa76844-7596-4283-9737-ee24b56f3202-istio-podinfo\") pod \"e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v\" (UID: \"9fa76844-7596-4283-9737-ee24b56f3202\") " pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v"
Jul 23 23:48:53.902282 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:53.902257    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cfb5\" (UniqueName: \"kubernetes.io/projected/9fa76844-7596-4283-9737-ee24b56f3202-kube-api-access-8cfb5\") pod \"e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v\" (UID: \"9fa76844-7596-4283-9737-ee24b56f3202\") " pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v"
Jul 23 23:48:53.902402 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:53.902264    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fa76844-7596-4283-9737-ee24b56f3202-istio-token\") pod \"e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v\" (UID: \"9fa76844-7596-4283-9737-ee24b56f3202\") " pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v"
Jul 23 23:48:54.007991 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:54.007887    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v"
Jul 23 23:48:54.146706 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:54.146670    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v"]
Jul 23 23:48:54.148609 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:48:54.148571    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fa76844_7596_4283_9737_ee24b56f3202.slice/crio-1c0d64796f7ae80cba9254216a30253b2f338ae657d5dfe17e04b870de56cb69 WatchSource:0}: Error finding container 1c0d64796f7ae80cba9254216a30253b2f338ae657d5dfe17e04b870de56cb69: Status 404 returned error can't find the container with id 1c0d64796f7ae80cba9254216a30253b2f338ae657d5dfe17e04b870de56cb69
Jul 23 23:48:54.150825 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:54.150783    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:48:54.150907 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:54.150850    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:48:54.150907 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:54.150883    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:48:54.291947 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:54.291847    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v" event={"ID":"9fa76844-7596-4283-9737-ee24b56f3202","Type":"ContainerStarted","Data":"59a4976460b4feac03624f783c7679be9bb5c6c64ee098d668b9f25383db9dab"}
Jul 23 23:48:54.291947 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:54.291906    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v" event={"ID":"9fa76844-7596-4283-9737-ee24b56f3202","Type":"ContainerStarted","Data":"1c0d64796f7ae80cba9254216a30253b2f338ae657d5dfe17e04b870de56cb69"}
Jul 23 23:48:54.312755 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:54.312696    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v" podStartSLOduration=1.3126807249999999 podStartE2EDuration="1.312680725s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:48:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:48:54.311298061 +0000 UTC m=+2123.680823746" watchObservedRunningTime="2026-07-23 23:48:54.312680725 +0000 UTC m=+2123.682206375"
Jul 23 23:48:55.008303 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:55.008267    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v"
Jul 23 23:48:55.012951 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:55.012925    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v"
Jul 23 23:48:55.297424 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:55.297342    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v"
Jul 23 23:48:55.298341 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:55.298324    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v"
Jul 23 23:48:56.293744 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:56.293705    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ce89b1-695bb8bf6c-ggkqf"]
Jul 23 23:48:56.294315 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:56.294290    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5492bdc3-62e0-43c9-b6e9-cb334db3a3ea" containerName="istio-proxy"
Jul 23 23:48:56.294315 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:56.294318    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5492bdc3-62e0-43c9-b6e9-cb334db3a3ea" containerName="istio-proxy"
Jul 23 23:48:56.298097 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:56.298075    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce89b1-695bb8bf6c-ggkqf"
Jul 23 23:48:56.300868 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:56.300847    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-ce89b1-serving-cert\""
Jul 23 23:48:56.308822 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:56.308797    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ce89b1-695bb8bf6c-ggkqf"]
Jul 23 23:48:56.317638 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:56.317610    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gp5s9\" (UniqueName: \"kubernetes.io/projected/4eca62b5-10c1-441a-95d1-db8c0d1fd065-kube-api-access-gp5s9\") pod \"maas-api-e2e-api-a-ce89b1-695bb8bf6c-ggkqf\" (UID: \"4eca62b5-10c1-441a-95d1-db8c0d1fd065\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce89b1-695bb8bf6c-ggkqf"
Jul 23 23:48:56.317812 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:56.317683    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4eca62b5-10c1-441a-95d1-db8c0d1fd065-openshift-service-ca\") pod \"maas-api-e2e-api-a-ce89b1-695bb8bf6c-ggkqf\" (UID: \"4eca62b5-10c1-441a-95d1-db8c0d1fd065\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce89b1-695bb8bf6c-ggkqf"
Jul 23 23:48:56.317812 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:56.317750    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4eca62b5-10c1-441a-95d1-db8c0d1fd065-maas-api-tls\") pod \"maas-api-e2e-api-a-ce89b1-695bb8bf6c-ggkqf\" (UID: \"4eca62b5-10c1-441a-95d1-db8c0d1fd065\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce89b1-695bb8bf6c-ggkqf"
Jul 23 23:48:56.418406 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:56.418374    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4eca62b5-10c1-441a-95d1-db8c0d1fd065-maas-api-tls\") pod \"maas-api-e2e-api-a-ce89b1-695bb8bf6c-ggkqf\" (UID: \"4eca62b5-10c1-441a-95d1-db8c0d1fd065\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce89b1-695bb8bf6c-ggkqf"
Jul 23 23:48:56.418605 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:56.418449    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gp5s9\" (UniqueName: \"kubernetes.io/projected/4eca62b5-10c1-441a-95d1-db8c0d1fd065-kube-api-access-gp5s9\") pod \"maas-api-e2e-api-a-ce89b1-695bb8bf6c-ggkqf\" (UID: \"4eca62b5-10c1-441a-95d1-db8c0d1fd065\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce89b1-695bb8bf6c-ggkqf"
Jul 23 23:48:56.418605 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:56.418483    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4eca62b5-10c1-441a-95d1-db8c0d1fd065-openshift-service-ca\") pod \"maas-api-e2e-api-a-ce89b1-695bb8bf6c-ggkqf\" (UID: \"4eca62b5-10c1-441a-95d1-db8c0d1fd065\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce89b1-695bb8bf6c-ggkqf"
Jul 23 23:48:56.418605 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:48:56.418546    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-ce89b1-serving-cert: secret "maas-api-e2e-api-a-ce89b1-serving-cert" not found
Jul 23 23:48:56.418785 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:48:56.418643    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4eca62b5-10c1-441a-95d1-db8c0d1fd065-maas-api-tls podName:4eca62b5-10c1-441a-95d1-db8c0d1fd065 nodeName:}" failed. No retries permitted until 2026-07-23 23:48:56.918619834 +0000 UTC m=+2126.288145472 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4eca62b5-10c1-441a-95d1-db8c0d1fd065-maas-api-tls") pod "maas-api-e2e-api-a-ce89b1-695bb8bf6c-ggkqf" (UID: "4eca62b5-10c1-441a-95d1-db8c0d1fd065") : secret "maas-api-e2e-api-a-ce89b1-serving-cert" not found
Jul 23 23:48:56.419159 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:56.419140    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4eca62b5-10c1-441a-95d1-db8c0d1fd065-openshift-service-ca\") pod \"maas-api-e2e-api-a-ce89b1-695bb8bf6c-ggkqf\" (UID: \"4eca62b5-10c1-441a-95d1-db8c0d1fd065\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce89b1-695bb8bf6c-ggkqf"
Jul 23 23:48:56.427353 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:56.427321    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gp5s9\" (UniqueName: \"kubernetes.io/projected/4eca62b5-10c1-441a-95d1-db8c0d1fd065-kube-api-access-gp5s9\") pod \"maas-api-e2e-api-a-ce89b1-695bb8bf6c-ggkqf\" (UID: \"4eca62b5-10c1-441a-95d1-db8c0d1fd065\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce89b1-695bb8bf6c-ggkqf"
Jul 23 23:48:56.924522 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:56.924478    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4eca62b5-10c1-441a-95d1-db8c0d1fd065-maas-api-tls\") pod \"maas-api-e2e-api-a-ce89b1-695bb8bf6c-ggkqf\" (UID: \"4eca62b5-10c1-441a-95d1-db8c0d1fd065\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce89b1-695bb8bf6c-ggkqf"
Jul 23 23:48:56.927166 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:56.927139    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4eca62b5-10c1-441a-95d1-db8c0d1fd065-maas-api-tls\") pod \"maas-api-e2e-api-a-ce89b1-695bb8bf6c-ggkqf\" (UID: \"4eca62b5-10c1-441a-95d1-db8c0d1fd065\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce89b1-695bb8bf6c-ggkqf"
Jul 23 23:48:57.210929 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:57.210886    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce89b1-695bb8bf6c-ggkqf"
Jul 23 23:48:57.557473 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:57.557447    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ce89b1-695bb8bf6c-ggkqf"]
Jul 23 23:48:57.559530 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:48:57.559500    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4eca62b5_10c1_441a_95d1_db8c0d1fd065.slice/crio-f648a9c2b3da3f555bb0d2397adc541bb147ce7785722be4090322cd7af9498b WatchSource:0}: Error finding container f648a9c2b3da3f555bb0d2397adc541bb147ce7785722be4090322cd7af9498b: Status 404 returned error can't find the container with id f648a9c2b3da3f555bb0d2397adc541bb147ce7785722be4090322cd7af9498b
Jul 23 23:48:58.052258 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:58.052236    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"
Jul 23 23:48:58.137266 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:58.137233    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-istio-token\") pod \"5ac35d4e-3dcf-4b95-80c8-90348e8c816a\" (UID: \"5ac35d4e-3dcf-4b95-80c8-90348e8c816a\") "
Jul 23 23:48:58.137419 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:58.137281    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2bngh\" (UniqueName: \"kubernetes.io/projected/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-kube-api-access-2bngh\") pod \"5ac35d4e-3dcf-4b95-80c8-90348e8c816a\" (UID: \"5ac35d4e-3dcf-4b95-80c8-90348e8c816a\") "
Jul 23 23:48:58.137419 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:58.137320    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-workload-certs\") pod \"5ac35d4e-3dcf-4b95-80c8-90348e8c816a\" (UID: \"5ac35d4e-3dcf-4b95-80c8-90348e8c816a\") "
Jul 23 23:48:58.137419 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:58.137354    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-istio-podinfo\") pod \"5ac35d4e-3dcf-4b95-80c8-90348e8c816a\" (UID: \"5ac35d4e-3dcf-4b95-80c8-90348e8c816a\") "
Jul 23 23:48:58.137419 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:58.137408    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-workload-socket\") pod \"5ac35d4e-3dcf-4b95-80c8-90348e8c816a\" (UID: \"5ac35d4e-3dcf-4b95-80c8-90348e8c816a\") "
Jul 23 23:48:58.137620 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:58.137431    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-istio-data\") pod \"5ac35d4e-3dcf-4b95-80c8-90348e8c816a\" (UID: \"5ac35d4e-3dcf-4b95-80c8-90348e8c816a\") "
Jul 23 23:48:58.137620 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:58.137536    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-credential-socket\") pod \"5ac35d4e-3dcf-4b95-80c8-90348e8c816a\" (UID: \"5ac35d4e-3dcf-4b95-80c8-90348e8c816a\") "
Jul 23 23:48:58.137620 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:58.137596    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-istiod-ca-cert\") pod \"5ac35d4e-3dcf-4b95-80c8-90348e8c816a\" (UID: \"5ac35d4e-3dcf-4b95-80c8-90348e8c816a\") "
Jul 23 23:48:58.137805 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:58.137636    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-istio-envoy\") pod \"5ac35d4e-3dcf-4b95-80c8-90348e8c816a\" (UID: \"5ac35d4e-3dcf-4b95-80c8-90348e8c816a\") "
Jul 23 23:48:58.137805 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:58.137647    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5ac35d4e-3dcf-4b95-80c8-90348e8c816a" (UID: "5ac35d4e-3dcf-4b95-80c8-90348e8c816a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:48:58.137805 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:58.137764    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5ac35d4e-3dcf-4b95-80c8-90348e8c816a" (UID: "5ac35d4e-3dcf-4b95-80c8-90348e8c816a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:48:58.137972 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:58.137909    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5ac35d4e-3dcf-4b95-80c8-90348e8c816a" (UID: "5ac35d4e-3dcf-4b95-80c8-90348e8c816a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:48:58.138263 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:58.138236    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-workload-certs\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:48:58.138553 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:58.138267    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-workload-socket\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:48:58.138553 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:58.138284    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-credential-socket\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:48:58.138553 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:58.138366    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5ac35d4e-3dcf-4b95-80c8-90348e8c816a" (UID: "5ac35d4e-3dcf-4b95-80c8-90348e8c816a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:48:58.138776 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:58.138545    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-istio-data" (OuterVolumeSpecName: "istio-data") pod "5ac35d4e-3dcf-4b95-80c8-90348e8c816a" (UID: "5ac35d4e-3dcf-4b95-80c8-90348e8c816a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:48:58.139516 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:58.139491    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-istio-token" (OuterVolumeSpecName: "istio-token") pod "5ac35d4e-3dcf-4b95-80c8-90348e8c816a" (UID: "5ac35d4e-3dcf-4b95-80c8-90348e8c816a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:48:58.139647 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:58.139620    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5ac35d4e-3dcf-4b95-80c8-90348e8c816a" (UID: "5ac35d4e-3dcf-4b95-80c8-90348e8c816a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 23:48:58.139768 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:58.139704    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-kube-api-access-2bngh" (OuterVolumeSpecName: "kube-api-access-2bngh") pod "5ac35d4e-3dcf-4b95-80c8-90348e8c816a" (UID: "5ac35d4e-3dcf-4b95-80c8-90348e8c816a"). InnerVolumeSpecName "kube-api-access-2bngh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:48:58.139932 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:58.139914    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5ac35d4e-3dcf-4b95-80c8-90348e8c816a" (UID: "5ac35d4e-3dcf-4b95-80c8-90348e8c816a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:48:58.239287 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:58.239255    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-istiod-ca-cert\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:48:58.239287 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:58.239284    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-istio-envoy\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:48:58.239287 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:58.239293    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-istio-token\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:48:58.239505 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:58.239301    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2bngh\" (UniqueName: \"kubernetes.io/projected/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-kube-api-access-2bngh\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:48:58.239505 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:58.239312    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-istio-podinfo\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:48:58.239505 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:58.239320    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ac35d4e-3dcf-4b95-80c8-90348e8c816a-istio-data\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:48:58.315692 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:58.315586    2578 generic.go:356] "Generic (PLEG): container finished" podID="5ac35d4e-3dcf-4b95-80c8-90348e8c816a" containerID="64d3a77107632083afb189dd620260eba8e0e4414424a121fccb1f205751dc64" exitCode=0
Jul 23 23:48:58.315692 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:58.315635    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"
Jul 23 23:48:58.315941 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:58.315688    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22" event={"ID":"5ac35d4e-3dcf-4b95-80c8-90348e8c816a","Type":"ContainerDied","Data":"64d3a77107632083afb189dd620260eba8e0e4414424a121fccb1f205751dc64"}
Jul 23 23:48:58.315941 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:58.315751    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22" event={"ID":"5ac35d4e-3dcf-4b95-80c8-90348e8c816a","Type":"ContainerDied","Data":"091b6975f95e134db65c3708580e92c27cab04cb9b96ea9cc43dd247e2b83184"}
Jul 23 23:48:58.315941 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:58.315782    2578 scope.go:117] "RemoveContainer" containerID="64d3a77107632083afb189dd620260eba8e0e4414424a121fccb1f205751dc64"
Jul 23 23:48:58.317349 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:58.317309    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce89b1-695bb8bf6c-ggkqf" event={"ID":"4eca62b5-10c1-441a-95d1-db8c0d1fd065","Type":"ContainerStarted","Data":"d8a1b6ca07e034d15025572f4d90822b63bb4143cdbb75ed01f96dade992c07f"}
Jul 23 23:48:58.317470 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:58.317364    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce89b1-695bb8bf6c-ggkqf" event={"ID":"4eca62b5-10c1-441a-95d1-db8c0d1fd065","Type":"ContainerStarted","Data":"f648a9c2b3da3f555bb0d2397adc541bb147ce7785722be4090322cd7af9498b"}
Jul 23 23:48:58.317470 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:58.317397    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce89b1-695bb8bf6c-ggkqf"
Jul 23 23:48:58.325251 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:58.325237    2578 scope.go:117] "RemoveContainer" containerID="2c39e25e5716bec948e77e3567d18787da9afa0b1463ed35c9bf03f9cb15a2ab"
Jul 23 23:48:58.333160 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:58.333142    2578 scope.go:117] "RemoveContainer" containerID="64d3a77107632083afb189dd620260eba8e0e4414424a121fccb1f205751dc64"
Jul 23 23:48:58.333510 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:48:58.333458    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64d3a77107632083afb189dd620260eba8e0e4414424a121fccb1f205751dc64\": container with ID starting with 64d3a77107632083afb189dd620260eba8e0e4414424a121fccb1f205751dc64 not found: ID does not exist" containerID="64d3a77107632083afb189dd620260eba8e0e4414424a121fccb1f205751dc64"
Jul 23 23:48:58.333510 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:58.333494    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64d3a77107632083afb189dd620260eba8e0e4414424a121fccb1f205751dc64"} err="failed to get container status \"64d3a77107632083afb189dd620260eba8e0e4414424a121fccb1f205751dc64\": rpc error: code = NotFound desc = could not find container \"64d3a77107632083afb189dd620260eba8e0e4414424a121fccb1f205751dc64\": container with ID starting with 64d3a77107632083afb189dd620260eba8e0e4414424a121fccb1f205751dc64 not found: ID does not exist"
Jul 23 23:48:58.333825 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:58.333519    2578 scope.go:117] "RemoveContainer" containerID="2c39e25e5716bec948e77e3567d18787da9afa0b1463ed35c9bf03f9cb15a2ab"
Jul 23 23:48:58.334035 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:48:58.334004    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c39e25e5716bec948e77e3567d18787da9afa0b1463ed35c9bf03f9cb15a2ab\": container with ID starting with 2c39e25e5716bec948e77e3567d18787da9afa0b1463ed35c9bf03f9cb15a2ab not found: ID does not exist" containerID="2c39e25e5716bec948e77e3567d18787da9afa0b1463ed35c9bf03f9cb15a2ab"
Jul 23 23:48:58.334140 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:58.334041    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c39e25e5716bec948e77e3567d18787da9afa0b1463ed35c9bf03f9cb15a2ab"} err="failed to get container status \"2c39e25e5716bec948e77e3567d18787da9afa0b1463ed35c9bf03f9cb15a2ab\": rpc error: code = NotFound desc = could not find container \"2c39e25e5716bec948e77e3567d18787da9afa0b1463ed35c9bf03f9cb15a2ab\": container with ID starting with 2c39e25e5716bec948e77e3567d18787da9afa0b1463ed35c9bf03f9cb15a2ab not found: ID does not exist"
Jul 23 23:48:58.335257 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:58.335219    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce89b1-695bb8bf6c-ggkqf" podStartSLOduration=1.84471377 podStartE2EDuration="2.335207236s" totalImagesPullingTime="490.493466ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:48:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:48:58.334442259 +0000 UTC m=+2127.703967918" watchObservedRunningTime="2026-07-23 23:48:58.335207236 +0000 UTC m=+2127.704732895"
Jul 23 23:48:58.347455 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:58.347431    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"]
Jul 23 23:48:58.353162 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:58.353140    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-d7e557-openshift-default-7cb9cc7d78-jjq22"]
Jul 23 23:48:59.237730 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:48:59.237691    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ac35d4e-3dcf-4b95-80c8-90348e8c816a" path="/var/lib/kubelet/pods/5ac35d4e-3dcf-4b95-80c8-90348e8c816a/volumes"
Jul 23 23:49:01.754158 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:01.754120    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm"]
Jul 23 23:49:01.754747 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:01.754726    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ac35d4e-3dcf-4b95-80c8-90348e8c816a" containerName="istio-proxy"
Jul 23 23:49:01.754812 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:01.754751    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ac35d4e-3dcf-4b95-80c8-90348e8c816a" containerName="istio-proxy"
Jul 23 23:49:01.754812 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:01.754779    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ac35d4e-3dcf-4b95-80c8-90348e8c816a" containerName="istio-proxy"
Jul 23 23:49:01.754812 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:01.754788    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ac35d4e-3dcf-4b95-80c8-90348e8c816a" containerName="istio-proxy"
Jul 23 23:49:01.754905 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:01.754875    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ac35d4e-3dcf-4b95-80c8-90348e8c816a" containerName="istio-proxy"
Jul 23 23:49:01.759803 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:01.759774    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm"
Jul 23 23:49:01.762743 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:01.762719    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-9d0833-openshift-default-dockercfg-j6jv2\""
Jul 23 23:49:01.770175 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:01.770149    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm"]
Jul 23 23:49:01.871837 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:01.871798    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7434c052-65e2-43a9-9e98-d049053ed1bb-istio-podinfo\") pod \"e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm\" (UID: \"7434c052-65e2-43a9-9e98-d049053ed1bb\") " pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm"
Jul 23 23:49:01.872017 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:01.871918    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7434c052-65e2-43a9-9e98-d049053ed1bb-istio-token\") pod \"e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm\" (UID: \"7434c052-65e2-43a9-9e98-d049053ed1bb\") " pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm"
Jul 23 23:49:01.872017 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:01.871952    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7434c052-65e2-43a9-9e98-d049053ed1bb-credential-socket\") pod \"e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm\" (UID: \"7434c052-65e2-43a9-9e98-d049053ed1bb\") " pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm"
Jul 23 23:49:01.872017 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:01.871994    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7434c052-65e2-43a9-9e98-d049053ed1bb-istiod-ca-cert\") pod \"e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm\" (UID: \"7434c052-65e2-43a9-9e98-d049053ed1bb\") " pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm"
Jul 23 23:49:01.872146 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:01.872096    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7434c052-65e2-43a9-9e98-d049053ed1bb-workload-socket\") pod \"e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm\" (UID: \"7434c052-65e2-43a9-9e98-d049053ed1bb\") " pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm"
Jul 23 23:49:01.872191 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:01.872149    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7434c052-65e2-43a9-9e98-d049053ed1bb-workload-certs\") pod \"e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm\" (UID: \"7434c052-65e2-43a9-9e98-d049053ed1bb\") " pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm"
Jul 23 23:49:01.872191 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:01.872177    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7434c052-65e2-43a9-9e98-d049053ed1bb-istio-data\") pod \"e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm\" (UID: \"7434c052-65e2-43a9-9e98-d049053ed1bb\") " pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm"
Jul 23 23:49:01.872260 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:01.872222    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7434c052-65e2-43a9-9e98-d049053ed1bb-istio-envoy\") pod \"e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm\" (UID: \"7434c052-65e2-43a9-9e98-d049053ed1bb\") " pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm"
Jul 23 23:49:01.872260 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:01.872238    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chdvs\" (UniqueName: \"kubernetes.io/projected/7434c052-65e2-43a9-9e98-d049053ed1bb-kube-api-access-chdvs\") pod \"e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm\" (UID: \"7434c052-65e2-43a9-9e98-d049053ed1bb\") " pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm"
Jul 23 23:49:01.973620 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:01.973578    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7434c052-65e2-43a9-9e98-d049053ed1bb-workload-socket\") pod \"e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm\" (UID: \"7434c052-65e2-43a9-9e98-d049053ed1bb\") " pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm"
Jul 23 23:49:01.973828 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:01.973637    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7434c052-65e2-43a9-9e98-d049053ed1bb-workload-certs\") pod \"e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm\" (UID: \"7434c052-65e2-43a9-9e98-d049053ed1bb\") " pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm"
Jul 23 23:49:01.973828 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:01.973690    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7434c052-65e2-43a9-9e98-d049053ed1bb-istio-data\") pod \"e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm\" (UID: \"7434c052-65e2-43a9-9e98-d049053ed1bb\") " pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm"
Jul 23 23:49:01.973828 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:01.973711    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7434c052-65e2-43a9-9e98-d049053ed1bb-istio-envoy\") pod \"e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm\" (UID: \"7434c052-65e2-43a9-9e98-d049053ed1bb\") " pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm"
Jul 23 23:49:01.973828 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:01.973728    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chdvs\" (UniqueName: \"kubernetes.io/projected/7434c052-65e2-43a9-9e98-d049053ed1bb-kube-api-access-chdvs\") pod \"e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm\" (UID: \"7434c052-65e2-43a9-9e98-d049053ed1bb\") " pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm"
Jul 23 23:49:01.973828 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:01.973760    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7434c052-65e2-43a9-9e98-d049053ed1bb-istio-podinfo\") pod \"e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm\" (UID: \"7434c052-65e2-43a9-9e98-d049053ed1bb\") " pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm"
Jul 23 23:49:01.973828 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:01.973825    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7434c052-65e2-43a9-9e98-d049053ed1bb-istio-token\") pod \"e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm\" (UID: \"7434c052-65e2-43a9-9e98-d049053ed1bb\") " pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm"
Jul 23 23:49:01.974124 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:01.973855    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7434c052-65e2-43a9-9e98-d049053ed1bb-credential-socket\") pod \"e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm\" (UID: \"7434c052-65e2-43a9-9e98-d049053ed1bb\") " pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm"
Jul 23 23:49:01.974124 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:01.973886    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7434c052-65e2-43a9-9e98-d049053ed1bb-istiod-ca-cert\") pod \"e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm\" (UID: \"7434c052-65e2-43a9-9e98-d049053ed1bb\") " pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm"
Jul 23 23:49:01.974124 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:01.974057    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7434c052-65e2-43a9-9e98-d049053ed1bb-workload-socket\") pod \"e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm\" (UID: \"7434c052-65e2-43a9-9e98-d049053ed1bb\") " pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm"
Jul 23 23:49:01.974252 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:01.974183    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7434c052-65e2-43a9-9e98-d049053ed1bb-workload-certs\") pod \"e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm\" (UID: \"7434c052-65e2-43a9-9e98-d049053ed1bb\") " pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm"
Jul 23 23:49:01.974332 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:01.974313    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7434c052-65e2-43a9-9e98-d049053ed1bb-credential-socket\") pod \"e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm\" (UID: \"7434c052-65e2-43a9-9e98-d049053ed1bb\") " pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm"
Jul 23 23:49:01.974382 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:01.974346    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7434c052-65e2-43a9-9e98-d049053ed1bb-istio-data\") pod \"e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm\" (UID: \"7434c052-65e2-43a9-9e98-d049053ed1bb\") " pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm"
Jul 23 23:49:01.974573 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:01.974547    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7434c052-65e2-43a9-9e98-d049053ed1bb-istiod-ca-cert\") pod \"e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm\" (UID: \"7434c052-65e2-43a9-9e98-d049053ed1bb\") " pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm"
Jul 23 23:49:01.976157 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:01.976138    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7434c052-65e2-43a9-9e98-d049053ed1bb-istio-envoy\") pod \"e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm\" (UID: \"7434c052-65e2-43a9-9e98-d049053ed1bb\") " pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm"
Jul 23 23:49:01.976360 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:01.976341    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7434c052-65e2-43a9-9e98-d049053ed1bb-istio-podinfo\") pod \"e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm\" (UID: \"7434c052-65e2-43a9-9e98-d049053ed1bb\") " pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm"
Jul 23 23:49:01.981849 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:01.981827    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7434c052-65e2-43a9-9e98-d049053ed1bb-istio-token\") pod \"e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm\" (UID: \"7434c052-65e2-43a9-9e98-d049053ed1bb\") " pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm"
Jul 23 23:49:01.982063 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:01.982044    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chdvs\" (UniqueName: \"kubernetes.io/projected/7434c052-65e2-43a9-9e98-d049053ed1bb-kube-api-access-chdvs\") pod \"e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm\" (UID: \"7434c052-65e2-43a9-9e98-d049053ed1bb\") " pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm"
Jul 23 23:49:02.071513 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:02.071408    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm"
Jul 23 23:49:02.215589 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:02.215558    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm"]
Jul 23 23:49:02.217951 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:49:02.217925    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7434c052_65e2_43a9_9e98_d049053ed1bb.slice/crio-3ee287408d82b36dd932a48d6bb8706bf46d14546c3eb8097daae4a3326ba62f WatchSource:0}: Error finding container 3ee287408d82b36dd932a48d6bb8706bf46d14546c3eb8097daae4a3326ba62f: Status 404 returned error can't find the container with id 3ee287408d82b36dd932a48d6bb8706bf46d14546c3eb8097daae4a3326ba62f
Jul 23 23:49:02.220282 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:02.220251    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:49:02.220350 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:02.220318    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:49:02.220388 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:02.220348    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:49:02.341997 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:02.341899    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm" event={"ID":"7434c052-65e2-43a9-9e98-d049053ed1bb","Type":"ContainerStarted","Data":"ca4e39970d0c83d2db455c7971dae59ec75691d131c831415ea3962851d17dc0"}
Jul 23 23:49:02.341997 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:02.341969    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm" event={"ID":"7434c052-65e2-43a9-9e98-d049053ed1bb","Type":"ContainerStarted","Data":"3ee287408d82b36dd932a48d6bb8706bf46d14546c3eb8097daae4a3326ba62f"}
Jul 23 23:49:02.364404 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:02.364352    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm" podStartSLOduration=1.36433718 podStartE2EDuration="1.36433718s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:49:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:49:02.362527338 +0000 UTC m=+2131.732052998" watchObservedRunningTime="2026-07-23 23:49:02.36433718 +0000 UTC m=+2131.733862839"
Jul 23 23:49:03.072097 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:03.072052    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm"
Jul 23 23:49:03.077545 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:03.077516    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm"
Jul 23 23:49:03.347687 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:03.347578    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm"
Jul 23 23:49:03.348774 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:03.348757    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm"
Jul 23 23:49:04.328840 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:04.328814    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce89b1-695bb8bf6c-ggkqf"
Jul 23 23:49:04.370413 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:04.370371    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9d0833-7fc7598765-v44f7"]
Jul 23 23:49:04.371162 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:04.371001    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ac35d4e-3dcf-4b95-80c8-90348e8c816a" containerName="istio-proxy"
Jul 23 23:49:04.374095 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:04.374071    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9d0833-7fc7598765-v44f7"
Jul 23 23:49:04.376840 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:04.376821    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-9d0833-serving-cert\""
Jul 23 23:49:04.385357 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:04.385334    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9d0833-7fc7598765-v44f7"]
Jul 23 23:49:04.502323 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:04.502283    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0-openshift-service-ca\") pod \"maas-api-e2e-api-b-9d0833-7fc7598765-v44f7\" (UID: \"56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9d0833-7fc7598765-v44f7"
Jul 23 23:49:04.502499 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:04.502332    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nb45\" (UniqueName: \"kubernetes.io/projected/56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0-kube-api-access-6nb45\") pod \"maas-api-e2e-api-b-9d0833-7fc7598765-v44f7\" (UID: \"56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9d0833-7fc7598765-v44f7"
Jul 23 23:49:04.502499 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:04.502435    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0-maas-api-tls\") pod \"maas-api-e2e-api-b-9d0833-7fc7598765-v44f7\" (UID: \"56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9d0833-7fc7598765-v44f7"
Jul 23 23:49:04.603340 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:04.603240    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0-openshift-service-ca\") pod \"maas-api-e2e-api-b-9d0833-7fc7598765-v44f7\" (UID: \"56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9d0833-7fc7598765-v44f7"
Jul 23 23:49:04.603340 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:04.603299    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nb45\" (UniqueName: \"kubernetes.io/projected/56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0-kube-api-access-6nb45\") pod \"maas-api-e2e-api-b-9d0833-7fc7598765-v44f7\" (UID: \"56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9d0833-7fc7598765-v44f7"
Jul 23 23:49:04.603340 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:04.603346    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0-maas-api-tls\") pod \"maas-api-e2e-api-b-9d0833-7fc7598765-v44f7\" (UID: \"56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9d0833-7fc7598765-v44f7"
Jul 23 23:49:04.603566 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:49:04.603482    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-9d0833-serving-cert: secret "maas-api-e2e-api-b-9d0833-serving-cert" not found
Jul 23 23:49:04.603566 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:49:04.603554    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0-maas-api-tls podName:56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0 nodeName:}" failed. No retries permitted until 2026-07-23 23:49:05.103532166 +0000 UTC m=+2134.473057818 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0-maas-api-tls") pod "maas-api-e2e-api-b-9d0833-7fc7598765-v44f7" (UID: "56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0") : secret "maas-api-e2e-api-b-9d0833-serving-cert" not found
Jul 23 23:49:04.603932 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:04.603912    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0-openshift-service-ca\") pod \"maas-api-e2e-api-b-9d0833-7fc7598765-v44f7\" (UID: \"56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9d0833-7fc7598765-v44f7"
Jul 23 23:49:04.612688 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:04.612665    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nb45\" (UniqueName: \"kubernetes.io/projected/56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0-kube-api-access-6nb45\") pod \"maas-api-e2e-api-b-9d0833-7fc7598765-v44f7\" (UID: \"56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9d0833-7fc7598765-v44f7"
Jul 23 23:49:05.107779 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:05.107741    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0-maas-api-tls\") pod \"maas-api-e2e-api-b-9d0833-7fc7598765-v44f7\" (UID: \"56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9d0833-7fc7598765-v44f7"
Jul 23 23:49:05.110384 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:05.110362    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0-maas-api-tls\") pod \"maas-api-e2e-api-b-9d0833-7fc7598765-v44f7\" (UID: \"56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9d0833-7fc7598765-v44f7"
Jul 23 23:49:05.288210 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:05.288150    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9d0833-7fc7598765-v44f7"
Jul 23 23:49:05.452558 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:05.452526    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9d0833-7fc7598765-v44f7"]
Jul 23 23:49:05.455893 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:49:05.455862    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56047b8e_4ad1_4f83_8abc_f5d8c0d98ad0.slice/crio-0a82ea708cee02a6d7f74715a074f189e17dee1532357fe47be9a2b740d73b5b WatchSource:0}: Error finding container 0a82ea708cee02a6d7f74715a074f189e17dee1532357fe47be9a2b740d73b5b: Status 404 returned error can't find the container with id 0a82ea708cee02a6d7f74715a074f189e17dee1532357fe47be9a2b740d73b5b
Jul 23 23:49:06.369805 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:06.369692    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9d0833-7fc7598765-v44f7" event={"ID":"56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0","Type":"ContainerStarted","Data":"843256e3b92d4ec0fddcf6f62a1de6e65fff783529ed147ad8888664bd88bd68"}
Jul 23 23:49:06.370077 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:06.370032    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9d0833-7fc7598765-v44f7" event={"ID":"56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0","Type":"ContainerStarted","Data":"0a82ea708cee02a6d7f74715a074f189e17dee1532357fe47be9a2b740d73b5b"}
Jul 23 23:49:06.371272 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:06.371255    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9d0833-7fc7598765-v44f7"
Jul 23 23:49:06.396442 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:06.396390    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9d0833-7fc7598765-v44f7" podStartSLOduration=1.865138977 podStartE2EDuration="2.396372545s" totalImagesPullingTime="531.233568ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:49:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:49:06.393829193 +0000 UTC m=+2135.763354844" watchObservedRunningTime="2026-07-23 23:49:06.396372545 +0000 UTC m=+2135.765898230"
Jul 23 23:49:11.338892 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:11.338837    2578 patch_prober.go:28] interesting pod/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 23 23:49:11.339304 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:11.338928    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v" podUID="9fa76844-7596-4283-9737-ee24b56f3202" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 23 23:49:12.409814 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:12.409726    2578 generic.go:356] "Generic (PLEG): container finished" podID="9fa76844-7596-4283-9737-ee24b56f3202" containerID="59a4976460b4feac03624f783c7679be9bb5c6c64ee098d668b9f25383db9dab" exitCode=0
Jul 23 23:49:12.409814 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:12.409771    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v" event={"ID":"9fa76844-7596-4283-9737-ee24b56f3202","Type":"ContainerDied","Data":"59a4976460b4feac03624f783c7679be9bb5c6c64ee098d668b9f25383db9dab"}
Jul 23 23:49:12.410249 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:12.410180    2578 scope.go:117] "RemoveContainer" containerID="59a4976460b4feac03624f783c7679be9bb5c6c64ee098d668b9f25383db9dab"
Jul 23 23:49:12.411234 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:12.411207    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:49:12.411290 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:12.411278    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:49:12.411330 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:12.411311    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:49:13.403731 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:13.403693    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9d0833-7fc7598765-v44f7"
Jul 23 23:49:13.421624 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:13.421490    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v" event={"ID":"9fa76844-7596-4283-9737-ee24b56f3202","Type":"ContainerStarted","Data":"2087af1853f8093d686d29ea23eba9fc23967cf3dd982fa62ac4d0ecb011e289"}
Jul 23 23:49:14.008670 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:14.008517    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v"
Jul 23 23:49:14.009341 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:14.008988    2578 patch_prober.go:28] interesting pod/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 23 23:49:14.009341 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:14.009044    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v" podUID="9fa76844-7596-4283-9737-ee24b56f3202" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 23 23:49:15.008599 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:15.008561    2578 patch_prober.go:28] interesting pod/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 23 23:49:15.009175 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:15.008618    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v" podUID="9fa76844-7596-4283-9737-ee24b56f3202" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 23 23:49:16.008451 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:16.008406    2578 patch_prober.go:28] interesting pod/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 23 23:49:16.008706 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:16.008469    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v" podUID="9fa76844-7596-4283-9737-ee24b56f3202" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 23 23:49:17.009307 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:17.009268    2578 patch_prober.go:28] interesting pod/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 23 23:49:17.009705 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:17.009333    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v" podUID="9fa76844-7596-4283-9737-ee24b56f3202" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 23 23:49:18.011949 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:18.011915    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v"
Jul 23 23:49:18.012382 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:18.012349    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v"
Jul 23 23:49:18.013200 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:18.013180    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v"
Jul 23 23:49:18.352451 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:18.352419    2578 patch_prober.go:28] interesting pod/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": read tcp 10.134.0.2:46132->10.134.0.50:15021: read: connection reset by peer" start-of-body=
Jul 23 23:49:18.352699 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:18.352636    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm" podUID="7434c052-65e2-43a9-9e98-d049053ed1bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": read tcp 10.134.0.2:46132->10.134.0.50:15021: read: connection reset by peer"
Jul 23 23:49:18.449391 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:18.449356    2578 generic.go:356] "Generic (PLEG): container finished" podID="7434c052-65e2-43a9-9e98-d049053ed1bb" containerID="ca4e39970d0c83d2db455c7971dae59ec75691d131c831415ea3962851d17dc0" exitCode=0
Jul 23 23:49:18.449559 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:18.449432    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm" event={"ID":"7434c052-65e2-43a9-9e98-d049053ed1bb","Type":"ContainerDied","Data":"ca4e39970d0c83d2db455c7971dae59ec75691d131c831415ea3962851d17dc0"}
Jul 23 23:49:18.449966 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:18.449947    2578 scope.go:117] "RemoveContainer" containerID="ca4e39970d0c83d2db455c7971dae59ec75691d131c831415ea3962851d17dc0"
Jul 23 23:49:18.450926 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:18.450897    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:49:18.451024 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:18.450965    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:49:18.451024 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:18.451003    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:49:19.456078 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:19.456045    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm" event={"ID":"7434c052-65e2-43a9-9e98-d049053ed1bb","Type":"ContainerStarted","Data":"944d0ed5d13bdde1d68bd9e20a392697378b425a2e63d2e3fb1c0b42a0ef3ca0"}
Jul 23 23:49:20.072085 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:20.072045    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm"
Jul 23 23:49:20.072542 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:20.072408    2578 patch_prober.go:28] interesting pod/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 23 23:49:20.072774 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:20.072748    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm" podUID="7434c052-65e2-43a9-9e98-d049053ed1bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 23 23:49:21.072797 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:21.072749    2578 patch_prober.go:28] interesting pod/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 23 23:49:21.073292 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:21.072823    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm" podUID="7434c052-65e2-43a9-9e98-d049053ed1bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 23 23:49:22.072139 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:22.072100    2578 patch_prober.go:28] interesting pod/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 23 23:49:22.072462 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:22.072424    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm" podUID="7434c052-65e2-43a9-9e98-d049053ed1bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 23 23:49:23.072675 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:23.072616    2578 patch_prober.go:28] interesting pod/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 23 23:49:23.073352 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:23.073317    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm" podUID="7434c052-65e2-43a9-9e98-d049053ed1bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 23 23:49:24.075980 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:24.075950    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm"
Jul 23 23:49:24.076432 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:24.076330    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm"
Jul 23 23:49:24.076959 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:24.076943    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm"
Jul 23 23:49:32.737368 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:32.737333    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ce89b1-695bb8bf6c-ggkqf"]
Jul 23 23:49:32.737867 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:32.737575    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce89b1-695bb8bf6c-ggkqf" podUID="4eca62b5-10c1-441a-95d1-db8c0d1fd065" containerName="maas-api" containerID="cri-o://d8a1b6ca07e034d15025572f4d90822b63bb4143cdbb75ed01f96dade992c07f" gracePeriod=30
Jul 23 23:49:32.988624 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:32.988562    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce89b1-695bb8bf6c-ggkqf"
Jul 23 23:49:33.013796 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:33.013766    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4eca62b5-10c1-441a-95d1-db8c0d1fd065-openshift-service-ca\") pod \"4eca62b5-10c1-441a-95d1-db8c0d1fd065\" (UID: \"4eca62b5-10c1-441a-95d1-db8c0d1fd065\") "
Jul 23 23:49:33.013951 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:33.013810    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gp5s9\" (UniqueName: \"kubernetes.io/projected/4eca62b5-10c1-441a-95d1-db8c0d1fd065-kube-api-access-gp5s9\") pod \"4eca62b5-10c1-441a-95d1-db8c0d1fd065\" (UID: \"4eca62b5-10c1-441a-95d1-db8c0d1fd065\") "
Jul 23 23:49:33.013951 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:33.013939    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4eca62b5-10c1-441a-95d1-db8c0d1fd065-maas-api-tls\") pod \"4eca62b5-10c1-441a-95d1-db8c0d1fd065\" (UID: \"4eca62b5-10c1-441a-95d1-db8c0d1fd065\") "
Jul 23 23:49:33.014251 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:33.014184    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4eca62b5-10c1-441a-95d1-db8c0d1fd065-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4eca62b5-10c1-441a-95d1-db8c0d1fd065" (UID: "4eca62b5-10c1-441a-95d1-db8c0d1fd065"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:49:33.014481 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:33.014445    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4eca62b5-10c1-441a-95d1-db8c0d1fd065-openshift-service-ca\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:49:33.016174 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:33.016144    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4eca62b5-10c1-441a-95d1-db8c0d1fd065-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4eca62b5-10c1-441a-95d1-db8c0d1fd065" (UID: "4eca62b5-10c1-441a-95d1-db8c0d1fd065"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 23:49:33.016263 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:33.016147    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4eca62b5-10c1-441a-95d1-db8c0d1fd065-kube-api-access-gp5s9" (OuterVolumeSpecName: "kube-api-access-gp5s9") pod "4eca62b5-10c1-441a-95d1-db8c0d1fd065" (UID: "4eca62b5-10c1-441a-95d1-db8c0d1fd065"). InnerVolumeSpecName "kube-api-access-gp5s9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:49:33.115634 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:33.115601    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gp5s9\" (UniqueName: \"kubernetes.io/projected/4eca62b5-10c1-441a-95d1-db8c0d1fd065-kube-api-access-gp5s9\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:49:33.115634 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:33.115628    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4eca62b5-10c1-441a-95d1-db8c0d1fd065-maas-api-tls\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:49:33.532467 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:33.532375    2578 generic.go:356] "Generic (PLEG): container finished" podID="4eca62b5-10c1-441a-95d1-db8c0d1fd065" containerID="d8a1b6ca07e034d15025572f4d90822b63bb4143cdbb75ed01f96dade992c07f" exitCode=0
Jul 23 23:49:33.532467 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:33.532434    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce89b1-695bb8bf6c-ggkqf"
Jul 23 23:49:33.532731 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:33.532462    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce89b1-695bb8bf6c-ggkqf" event={"ID":"4eca62b5-10c1-441a-95d1-db8c0d1fd065","Type":"ContainerDied","Data":"d8a1b6ca07e034d15025572f4d90822b63bb4143cdbb75ed01f96dade992c07f"}
Jul 23 23:49:33.532731 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:33.532526    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ce89b1-695bb8bf6c-ggkqf" event={"ID":"4eca62b5-10c1-441a-95d1-db8c0d1fd065","Type":"ContainerDied","Data":"f648a9c2b3da3f555bb0d2397adc541bb147ce7785722be4090322cd7af9498b"}
Jul 23 23:49:33.532731 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:33.532554    2578 scope.go:117] "RemoveContainer" containerID="d8a1b6ca07e034d15025572f4d90822b63bb4143cdbb75ed01f96dade992c07f"
Jul 23 23:49:33.541763 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:33.541741    2578 scope.go:117] "RemoveContainer" containerID="d8a1b6ca07e034d15025572f4d90822b63bb4143cdbb75ed01f96dade992c07f"
Jul 23 23:49:33.542025 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:49:33.542008    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8a1b6ca07e034d15025572f4d90822b63bb4143cdbb75ed01f96dade992c07f\": container with ID starting with d8a1b6ca07e034d15025572f4d90822b63bb4143cdbb75ed01f96dade992c07f not found: ID does not exist" containerID="d8a1b6ca07e034d15025572f4d90822b63bb4143cdbb75ed01f96dade992c07f"
Jul 23 23:49:33.542081 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:33.542033    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8a1b6ca07e034d15025572f4d90822b63bb4143cdbb75ed01f96dade992c07f"} err="failed to get container status \"d8a1b6ca07e034d15025572f4d90822b63bb4143cdbb75ed01f96dade992c07f\": rpc error: code = NotFound desc = could not find container \"d8a1b6ca07e034d15025572f4d90822b63bb4143cdbb75ed01f96dade992c07f\": container with ID starting with d8a1b6ca07e034d15025572f4d90822b63bb4143cdbb75ed01f96dade992c07f not found: ID does not exist"
Jul 23 23:49:33.549364 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:33.549335    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ce89b1-695bb8bf6c-ggkqf"]
Jul 23 23:49:33.552596 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:33.552569    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ce89b1-695bb8bf6c-ggkqf"]
Jul 23 23:49:35.236732 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:35.236700    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4eca62b5-10c1-441a-95d1-db8c0d1fd065" path="/var/lib/kubelet/pods/4eca62b5-10c1-441a-95d1-db8c0d1fd065/volumes"
Jul 23 23:49:43.607612 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:43.607575    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v"]
Jul 23 23:49:43.608143 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:43.607831    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v" podUID="9fa76844-7596-4283-9737-ee24b56f3202" containerName="istio-proxy" containerID="cri-o://2087af1853f8093d686d29ea23eba9fc23967cf3dd982fa62ac4d0ecb011e289" gracePeriod=30
Jul 23 23:49:48.014238 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:48.014195    2578 patch_prober.go:28] interesting pod/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 23 23:49:48.014632 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:48.014252    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v" podUID="9fa76844-7596-4283-9737-ee24b56f3202" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 23 23:49:48.857230 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:48.857205    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v"
Jul 23 23:49:48.981079 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:48.981050    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fa76844-7596-4283-9737-ee24b56f3202-workload-certs\") pod \"9fa76844-7596-4283-9737-ee24b56f3202\" (UID: \"9fa76844-7596-4283-9737-ee24b56f3202\") "
Jul 23 23:49:48.981256 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:48.981087    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fa76844-7596-4283-9737-ee24b56f3202-istiod-ca-cert\") pod \"9fa76844-7596-4283-9737-ee24b56f3202\" (UID: \"9fa76844-7596-4283-9737-ee24b56f3202\") "
Jul 23 23:49:48.981256 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:48.981127    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fa76844-7596-4283-9737-ee24b56f3202-istio-token\") pod \"9fa76844-7596-4283-9737-ee24b56f3202\" (UID: \"9fa76844-7596-4283-9737-ee24b56f3202\") "
Jul 23 23:49:48.981256 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:48.981168    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8cfb5\" (UniqueName: \"kubernetes.io/projected/9fa76844-7596-4283-9737-ee24b56f3202-kube-api-access-8cfb5\") pod \"9fa76844-7596-4283-9737-ee24b56f3202\" (UID: \"9fa76844-7596-4283-9737-ee24b56f3202\") "
Jul 23 23:49:48.981256 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:48.981234    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fa76844-7596-4283-9737-ee24b56f3202-workload-socket\") pod \"9fa76844-7596-4283-9737-ee24b56f3202\" (UID: \"9fa76844-7596-4283-9737-ee24b56f3202\") "
Jul 23 23:49:48.981490 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:48.981272    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fa76844-7596-4283-9737-ee24b56f3202-istio-data\") pod \"9fa76844-7596-4283-9737-ee24b56f3202\" (UID: \"9fa76844-7596-4283-9737-ee24b56f3202\") "
Jul 23 23:49:48.981490 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:48.981296    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fa76844-7596-4283-9737-ee24b56f3202-credential-socket\") pod \"9fa76844-7596-4283-9737-ee24b56f3202\" (UID: \"9fa76844-7596-4283-9737-ee24b56f3202\") "
Jul 23 23:49:48.981490 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:48.981350    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fa76844-7596-4283-9737-ee24b56f3202-istio-envoy\") pod \"9fa76844-7596-4283-9737-ee24b56f3202\" (UID: \"9fa76844-7596-4283-9737-ee24b56f3202\") "
Jul 23 23:49:48.981490 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:48.981381    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fa76844-7596-4283-9737-ee24b56f3202-istio-podinfo\") pod \"9fa76844-7596-4283-9737-ee24b56f3202\" (UID: \"9fa76844-7596-4283-9737-ee24b56f3202\") "
Jul 23 23:49:48.981490 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:48.981387    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fa76844-7596-4283-9737-ee24b56f3202-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9fa76844-7596-4283-9737-ee24b56f3202" (UID: "9fa76844-7596-4283-9737-ee24b56f3202"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:49:48.981732 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:48.981574    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fa76844-7596-4283-9737-ee24b56f3202-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9fa76844-7596-4283-9737-ee24b56f3202" (UID: "9fa76844-7596-4283-9737-ee24b56f3202"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:49:48.981732 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:48.981581    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fa76844-7596-4283-9737-ee24b56f3202-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9fa76844-7596-4283-9737-ee24b56f3202" (UID: "9fa76844-7596-4283-9737-ee24b56f3202"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:49:48.981732 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:48.981620    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fa76844-7596-4283-9737-ee24b56f3202-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9fa76844-7596-4283-9737-ee24b56f3202" (UID: "9fa76844-7596-4283-9737-ee24b56f3202"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:49:48.981871 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:48.981796    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fa76844-7596-4283-9737-ee24b56f3202-workload-certs\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:49:48.981871 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:48.981815    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fa76844-7596-4283-9737-ee24b56f3202-istiod-ca-cert\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:49:48.981871 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:48.981829    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fa76844-7596-4283-9737-ee24b56f3202-workload-socket\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:49:48.981871 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:48.981844    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fa76844-7596-4283-9737-ee24b56f3202-credential-socket\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:49:48.982358 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:48.982328    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fa76844-7596-4283-9737-ee24b56f3202-istio-data" (OuterVolumeSpecName: "istio-data") pod "9fa76844-7596-4283-9737-ee24b56f3202" (UID: "9fa76844-7596-4283-9737-ee24b56f3202"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:49:48.983606 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:48.983575    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fa76844-7596-4283-9737-ee24b56f3202-kube-api-access-8cfb5" (OuterVolumeSpecName: "kube-api-access-8cfb5") pod "9fa76844-7596-4283-9737-ee24b56f3202" (UID: "9fa76844-7596-4283-9737-ee24b56f3202"). InnerVolumeSpecName "kube-api-access-8cfb5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:49:48.983789 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:48.983764    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fa76844-7596-4283-9737-ee24b56f3202-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9fa76844-7596-4283-9737-ee24b56f3202" (UID: "9fa76844-7596-4283-9737-ee24b56f3202"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:49:48.983968 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:48.983945    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fa76844-7596-4283-9737-ee24b56f3202-istio-token" (OuterVolumeSpecName: "istio-token") pod "9fa76844-7596-4283-9737-ee24b56f3202" (UID: "9fa76844-7596-4283-9737-ee24b56f3202"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:49:48.984077 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:48.984062    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9fa76844-7596-4283-9737-ee24b56f3202-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9fa76844-7596-4283-9737-ee24b56f3202" (UID: "9fa76844-7596-4283-9737-ee24b56f3202"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 23:49:49.083140 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:49.083105    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fa76844-7596-4283-9737-ee24b56f3202-istio-token\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:49:49.083140 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:49.083132    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8cfb5\" (UniqueName: \"kubernetes.io/projected/9fa76844-7596-4283-9737-ee24b56f3202-kube-api-access-8cfb5\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:49:49.083140 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:49.083144    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fa76844-7596-4283-9737-ee24b56f3202-istio-data\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:49:49.083555 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:49.083152    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fa76844-7596-4283-9737-ee24b56f3202-istio-envoy\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:49:49.083555 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:49.083160    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fa76844-7596-4283-9737-ee24b56f3202-istio-podinfo\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:49:49.619115 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:49.619014    2578 generic.go:356] "Generic (PLEG): container finished" podID="9fa76844-7596-4283-9737-ee24b56f3202" containerID="2087af1853f8093d686d29ea23eba9fc23967cf3dd982fa62ac4d0ecb011e289" exitCode=0
Jul 23 23:49:49.619115 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:49.619071    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v"
Jul 23 23:49:49.619360 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:49.619103    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v" event={"ID":"9fa76844-7596-4283-9737-ee24b56f3202","Type":"ContainerDied","Data":"2087af1853f8093d686d29ea23eba9fc23967cf3dd982fa62ac4d0ecb011e289"}
Jul 23 23:49:49.619360 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:49.619174    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v" event={"ID":"9fa76844-7596-4283-9737-ee24b56f3202","Type":"ContainerDied","Data":"1c0d64796f7ae80cba9254216a30253b2f338ae657d5dfe17e04b870de56cb69"}
Jul 23 23:49:49.619360 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:49.619204    2578 scope.go:117] "RemoveContainer" containerID="2087af1853f8093d686d29ea23eba9fc23967cf3dd982fa62ac4d0ecb011e289"
Jul 23 23:49:49.628037 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:49.628015    2578 scope.go:117] "RemoveContainer" containerID="59a4976460b4feac03624f783c7679be9bb5c6c64ee098d668b9f25383db9dab"
Jul 23 23:49:49.636385 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:49.636365    2578 scope.go:117] "RemoveContainer" containerID="2087af1853f8093d686d29ea23eba9fc23967cf3dd982fa62ac4d0ecb011e289"
Jul 23 23:49:49.636736 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:49:49.636645    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2087af1853f8093d686d29ea23eba9fc23967cf3dd982fa62ac4d0ecb011e289\": container with ID starting with 2087af1853f8093d686d29ea23eba9fc23967cf3dd982fa62ac4d0ecb011e289 not found: ID does not exist" containerID="2087af1853f8093d686d29ea23eba9fc23967cf3dd982fa62ac4d0ecb011e289"
Jul 23 23:49:49.636812 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:49.636751    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2087af1853f8093d686d29ea23eba9fc23967cf3dd982fa62ac4d0ecb011e289"} err="failed to get container status \"2087af1853f8093d686d29ea23eba9fc23967cf3dd982fa62ac4d0ecb011e289\": rpc error: code = NotFound desc = could not find container \"2087af1853f8093d686d29ea23eba9fc23967cf3dd982fa62ac4d0ecb011e289\": container with ID starting with 2087af1853f8093d686d29ea23eba9fc23967cf3dd982fa62ac4d0ecb011e289 not found: ID does not exist"
Jul 23 23:49:49.636812 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:49.636776    2578 scope.go:117] "RemoveContainer" containerID="59a4976460b4feac03624f783c7679be9bb5c6c64ee098d668b9f25383db9dab"
Jul 23 23:49:49.637038 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:49:49.637022    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59a4976460b4feac03624f783c7679be9bb5c6c64ee098d668b9f25383db9dab\": container with ID starting with 59a4976460b4feac03624f783c7679be9bb5c6c64ee098d668b9f25383db9dab not found: ID does not exist" containerID="59a4976460b4feac03624f783c7679be9bb5c6c64ee098d668b9f25383db9dab"
Jul 23 23:49:49.637079 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:49.637043    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59a4976460b4feac03624f783c7679be9bb5c6c64ee098d668b9f25383db9dab"} err="failed to get container status \"59a4976460b4feac03624f783c7679be9bb5c6c64ee098d668b9f25383db9dab\": rpc error: code = NotFound desc = could not find container \"59a4976460b4feac03624f783c7679be9bb5c6c64ee098d668b9f25383db9dab\": container with ID starting with 59a4976460b4feac03624f783c7679be9bb5c6c64ee098d668b9f25383db9dab not found: ID does not exist"
Jul 23 23:49:49.638194 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:49.638176    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v"]
Jul 23 23:49:49.641489 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:49.641468    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-ce89b1-openshift-default-54459f4564-wsk2v"]
Jul 23 23:49:51.236375 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:51.236338    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9fa76844-7596-4283-9737-ee24b56f3202" path="/var/lib/kubelet/pods/9fa76844-7596-4283-9737-ee24b56f3202/volumes"
Jul 23 23:49:59.399152 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:59.399113    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9d0833-7fc7598765-v44f7"]
Jul 23 23:49:59.399718 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:59.399391    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9d0833-7fc7598765-v44f7" podUID="56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0" containerName="maas-api" containerID="cri-o://843256e3b92d4ec0fddcf6f62a1de6e65fff783529ed147ad8888664bd88bd68" gracePeriod=30
Jul 23 23:49:59.649486 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:59.649428    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9d0833-7fc7598765-v44f7"
Jul 23 23:49:59.672323 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:59.672291    2578 generic.go:356] "Generic (PLEG): container finished" podID="56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0" containerID="843256e3b92d4ec0fddcf6f62a1de6e65fff783529ed147ad8888664bd88bd68" exitCode=0
Jul 23 23:49:59.672490 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:59.672334    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9d0833-7fc7598765-v44f7"
Jul 23 23:49:59.672490 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:59.672360    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9d0833-7fc7598765-v44f7" event={"ID":"56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0","Type":"ContainerDied","Data":"843256e3b92d4ec0fddcf6f62a1de6e65fff783529ed147ad8888664bd88bd68"}
Jul 23 23:49:59.672490 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:59.672410    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9d0833-7fc7598765-v44f7" event={"ID":"56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0","Type":"ContainerDied","Data":"0a82ea708cee02a6d7f74715a074f189e17dee1532357fe47be9a2b740d73b5b"}
Jul 23 23:49:59.672490 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:59.672440    2578 scope.go:117] "RemoveContainer" containerID="843256e3b92d4ec0fddcf6f62a1de6e65fff783529ed147ad8888664bd88bd68"
Jul 23 23:49:59.681191 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:59.681170    2578 scope.go:117] "RemoveContainer" containerID="843256e3b92d4ec0fddcf6f62a1de6e65fff783529ed147ad8888664bd88bd68"
Jul 23 23:49:59.681465 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:49:59.681446    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"843256e3b92d4ec0fddcf6f62a1de6e65fff783529ed147ad8888664bd88bd68\": container with ID starting with 843256e3b92d4ec0fddcf6f62a1de6e65fff783529ed147ad8888664bd88bd68 not found: ID does not exist" containerID="843256e3b92d4ec0fddcf6f62a1de6e65fff783529ed147ad8888664bd88bd68"
Jul 23 23:49:59.681532 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:59.681476    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"843256e3b92d4ec0fddcf6f62a1de6e65fff783529ed147ad8888664bd88bd68"} err="failed to get container status \"843256e3b92d4ec0fddcf6f62a1de6e65fff783529ed147ad8888664bd88bd68\": rpc error: code = NotFound desc = could not find container \"843256e3b92d4ec0fddcf6f62a1de6e65fff783529ed147ad8888664bd88bd68\": container with ID starting with 843256e3b92d4ec0fddcf6f62a1de6e65fff783529ed147ad8888664bd88bd68 not found: ID does not exist"
Jul 23 23:49:59.686827 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:59.686808    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0-openshift-service-ca\") pod \"56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0\" (UID: \"56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0\") "
Jul 23 23:49:59.686928 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:59.686854    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6nb45\" (UniqueName: \"kubernetes.io/projected/56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0-kube-api-access-6nb45\") pod \"56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0\" (UID: \"56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0\") "
Jul 23 23:49:59.686928 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:59.686899    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0-maas-api-tls\") pod \"56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0\" (UID: \"56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0\") "
Jul 23 23:49:59.687188 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:59.687160    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0" (UID: "56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:49:59.689151 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:59.689121    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0-kube-api-access-6nb45" (OuterVolumeSpecName: "kube-api-access-6nb45") pod "56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0" (UID: "56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0"). InnerVolumeSpecName "kube-api-access-6nb45". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:49:59.689151 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:59.689131    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0" (UID: "56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 23:49:59.787907 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:59.787869    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0-openshift-service-ca\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:49:59.787907 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:59.787901    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6nb45\" (UniqueName: \"kubernetes.io/projected/56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0-kube-api-access-6nb45\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:49:59.787907 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:59.787914    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0-maas-api-tls\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:49:59.996359 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:49:59.996315    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9d0833-7fc7598765-v44f7"]
Jul 23 23:50:00.000988 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:00.000952    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9d0833-7fc7598765-v44f7"]
Jul 23 23:50:01.236479 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:01.236439    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0" path="/var/lib/kubelet/pods/56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0/volumes"
Jul 23 23:50:10.368645 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:10.368607    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm"]
Jul 23 23:50:10.369136 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:10.368986    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm" podUID="7434c052-65e2-43a9-9e98-d049053ed1bb" containerName="istio-proxy" containerID="cri-o://944d0ed5d13bdde1d68bd9e20a392697378b425a2e63d2e3fb1c0b42a0ef3ca0" gracePeriod=30
Jul 23 23:50:11.298514 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.298478    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8"]
Jul 23 23:50:11.299043 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.299026    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4eca62b5-10c1-441a-95d1-db8c0d1fd065" containerName="maas-api"
Jul 23 23:50:11.299101 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.299047    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4eca62b5-10c1-441a-95d1-db8c0d1fd065" containerName="maas-api"
Jul 23 23:50:11.299101 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.299067    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0" containerName="maas-api"
Jul 23 23:50:11.299101 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.299076    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0" containerName="maas-api"
Jul 23 23:50:11.299101 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.299088    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fa76844-7596-4283-9737-ee24b56f3202" containerName="istio-proxy"
Jul 23 23:50:11.299101 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.299096    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fa76844-7596-4283-9737-ee24b56f3202" containerName="istio-proxy"
Jul 23 23:50:11.299340 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.299123    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fa76844-7596-4283-9737-ee24b56f3202" containerName="istio-proxy"
Jul 23 23:50:11.299340 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.299132    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fa76844-7596-4283-9737-ee24b56f3202" containerName="istio-proxy"
Jul 23 23:50:11.299340 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.299218    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4eca62b5-10c1-441a-95d1-db8c0d1fd065" containerName="maas-api"
Jul 23 23:50:11.299340 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.299228    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fa76844-7596-4283-9737-ee24b56f3202" containerName="istio-proxy"
Jul 23 23:50:11.299340 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.299237    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="56047b8e-4ad1-4f83-8abc-f5d8c0d98ad0" containerName="maas-api"
Jul 23 23:50:11.299340 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.299250    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fa76844-7596-4283-9737-ee24b56f3202" containerName="istio-proxy"
Jul 23 23:50:11.302456 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.302437    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8"
Jul 23 23:50:11.305087 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.305067    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-867102-openshift-default-dockercfg-pxxfq\""
Jul 23 23:50:11.312597 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.312574    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8"]
Jul 23 23:50:11.389090 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.389054    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13b2835b-0e85-44a3-a4b1-9172fff9f137-istio-data\") pod \"e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8\" (UID: \"13b2835b-0e85-44a3-a4b1-9172fff9f137\") " pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8"
Jul 23 23:50:11.389502 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.389108    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13b2835b-0e85-44a3-a4b1-9172fff9f137-credential-socket\") pod \"e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8\" (UID: \"13b2835b-0e85-44a3-a4b1-9172fff9f137\") " pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8"
Jul 23 23:50:11.389502 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.389168    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13b2835b-0e85-44a3-a4b1-9172fff9f137-istiod-ca-cert\") pod \"e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8\" (UID: \"13b2835b-0e85-44a3-a4b1-9172fff9f137\") " pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8"
Jul 23 23:50:11.389502 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.389194    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13b2835b-0e85-44a3-a4b1-9172fff9f137-istio-podinfo\") pod \"e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8\" (UID: \"13b2835b-0e85-44a3-a4b1-9172fff9f137\") " pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8"
Jul 23 23:50:11.389502 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.389239    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6srdt\" (UniqueName: \"kubernetes.io/projected/13b2835b-0e85-44a3-a4b1-9172fff9f137-kube-api-access-6srdt\") pod \"e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8\" (UID: \"13b2835b-0e85-44a3-a4b1-9172fff9f137\") " pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8"
Jul 23 23:50:11.389502 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.389271    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13b2835b-0e85-44a3-a4b1-9172fff9f137-istio-token\") pod \"e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8\" (UID: \"13b2835b-0e85-44a3-a4b1-9172fff9f137\") " pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8"
Jul 23 23:50:11.389502 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.389296    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13b2835b-0e85-44a3-a4b1-9172fff9f137-workload-certs\") pod \"e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8\" (UID: \"13b2835b-0e85-44a3-a4b1-9172fff9f137\") " pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8"
Jul 23 23:50:11.389502 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.389323    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13b2835b-0e85-44a3-a4b1-9172fff9f137-workload-socket\") pod \"e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8\" (UID: \"13b2835b-0e85-44a3-a4b1-9172fff9f137\") " pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8"
Jul 23 23:50:11.389502 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.389409    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13b2835b-0e85-44a3-a4b1-9172fff9f137-istio-envoy\") pod \"e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8\" (UID: \"13b2835b-0e85-44a3-a4b1-9172fff9f137\") " pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8"
Jul 23 23:50:11.490672 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.490619    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13b2835b-0e85-44a3-a4b1-9172fff9f137-istiod-ca-cert\") pod \"e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8\" (UID: \"13b2835b-0e85-44a3-a4b1-9172fff9f137\") " pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8"
Jul 23 23:50:11.490857 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.490712    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13b2835b-0e85-44a3-a4b1-9172fff9f137-istio-podinfo\") pod \"e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8\" (UID: \"13b2835b-0e85-44a3-a4b1-9172fff9f137\") " pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8"
Jul 23 23:50:11.490857 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.490765    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6srdt\" (UniqueName: \"kubernetes.io/projected/13b2835b-0e85-44a3-a4b1-9172fff9f137-kube-api-access-6srdt\") pod \"e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8\" (UID: \"13b2835b-0e85-44a3-a4b1-9172fff9f137\") " pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8"
Jul 23 23:50:11.490857 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.490795    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13b2835b-0e85-44a3-a4b1-9172fff9f137-istio-token\") pod \"e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8\" (UID: \"13b2835b-0e85-44a3-a4b1-9172fff9f137\") " pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8"
Jul 23 23:50:11.490857 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.490836    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13b2835b-0e85-44a3-a4b1-9172fff9f137-workload-certs\") pod \"e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8\" (UID: \"13b2835b-0e85-44a3-a4b1-9172fff9f137\") " pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8"
Jul 23 23:50:11.491075 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.490874    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13b2835b-0e85-44a3-a4b1-9172fff9f137-workload-socket\") pod \"e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8\" (UID: \"13b2835b-0e85-44a3-a4b1-9172fff9f137\") " pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8"
Jul 23 23:50:11.491075 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.490921    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13b2835b-0e85-44a3-a4b1-9172fff9f137-istio-envoy\") pod \"e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8\" (UID: \"13b2835b-0e85-44a3-a4b1-9172fff9f137\") " pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8"
Jul 23 23:50:11.491075 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.490985    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13b2835b-0e85-44a3-a4b1-9172fff9f137-istio-data\") pod \"e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8\" (UID: \"13b2835b-0e85-44a3-a4b1-9172fff9f137\") " pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8"
Jul 23 23:50:11.491075 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.491028    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13b2835b-0e85-44a3-a4b1-9172fff9f137-credential-socket\") pod \"e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8\" (UID: \"13b2835b-0e85-44a3-a4b1-9172fff9f137\") " pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8"
Jul 23 23:50:11.491601 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.491571    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13b2835b-0e85-44a3-a4b1-9172fff9f137-workload-socket\") pod \"e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8\" (UID: \"13b2835b-0e85-44a3-a4b1-9172fff9f137\") " pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8"
Jul 23 23:50:11.491871 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.491847    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13b2835b-0e85-44a3-a4b1-9172fff9f137-workload-certs\") pod \"e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8\" (UID: \"13b2835b-0e85-44a3-a4b1-9172fff9f137\") " pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8"
Jul 23 23:50:11.491960 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.491919    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13b2835b-0e85-44a3-a4b1-9172fff9f137-credential-socket\") pod \"e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8\" (UID: \"13b2835b-0e85-44a3-a4b1-9172fff9f137\") " pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8"
Jul 23 23:50:11.492019 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.491961    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13b2835b-0e85-44a3-a4b1-9172fff9f137-istiod-ca-cert\") pod \"e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8\" (UID: \"13b2835b-0e85-44a3-a4b1-9172fff9f137\") " pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8"
Jul 23 23:50:11.492151 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.492131    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13b2835b-0e85-44a3-a4b1-9172fff9f137-istio-data\") pod \"e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8\" (UID: \"13b2835b-0e85-44a3-a4b1-9172fff9f137\") " pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8"
Jul 23 23:50:11.494367 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.494345    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13b2835b-0e85-44a3-a4b1-9172fff9f137-istio-envoy\") pod \"e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8\" (UID: \"13b2835b-0e85-44a3-a4b1-9172fff9f137\") " pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8"
Jul 23 23:50:11.494862 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.494844    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13b2835b-0e85-44a3-a4b1-9172fff9f137-istio-podinfo\") pod \"e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8\" (UID: \"13b2835b-0e85-44a3-a4b1-9172fff9f137\") " pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8"
Jul 23 23:50:11.499916 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.499886    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6srdt\" (UniqueName: \"kubernetes.io/projected/13b2835b-0e85-44a3-a4b1-9172fff9f137-kube-api-access-6srdt\") pod \"e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8\" (UID: \"13b2835b-0e85-44a3-a4b1-9172fff9f137\") " pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8"
Jul 23 23:50:11.501305 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.501270    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13b2835b-0e85-44a3-a4b1-9172fff9f137-istio-token\") pod \"e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8\" (UID: \"13b2835b-0e85-44a3-a4b1-9172fff9f137\") " pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8"
Jul 23 23:50:11.613120 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.613025    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8"
Jul 23 23:50:11.759227 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.759197    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8"]
Jul 23 23:50:11.761860 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:50:11.761836    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13b2835b_0e85_44a3_a4b1_9172fff9f137.slice/crio-581d63b11c0ba7bd2e4aefa149d2c789fb375c71f1a73ce0d05c9cbb207eb658 WatchSource:0}: Error finding container 581d63b11c0ba7bd2e4aefa149d2c789fb375c71f1a73ce0d05c9cbb207eb658: Status 404 returned error can't find the container with id 581d63b11c0ba7bd2e4aefa149d2c789fb375c71f1a73ce0d05c9cbb207eb658
Jul 23 23:50:11.764048 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.764009    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:50:11.764117 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.764086    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:50:11.764154 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:11.764115    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:50:12.736077 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:12.736038    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8" event={"ID":"13b2835b-0e85-44a3-a4b1-9172fff9f137","Type":"ContainerStarted","Data":"92108de625269ac67a2674374b54b817f3c8e6821bac437554ac7dc168ad52a7"}
Jul 23 23:50:12.736509 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:12.736091    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8" event={"ID":"13b2835b-0e85-44a3-a4b1-9172fff9f137","Type":"ContainerStarted","Data":"581d63b11c0ba7bd2e4aefa149d2c789fb375c71f1a73ce0d05c9cbb207eb658"}
Jul 23 23:50:12.757823 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:12.757761    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8" podStartSLOduration=1.7577405179999999 podStartE2EDuration="1.757740518s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:50:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:50:12.756973253 +0000 UTC m=+2202.126498931" watchObservedRunningTime="2026-07-23 23:50:12.757740518 +0000 UTC m=+2202.127266189"
Jul 23 23:50:13.613745 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:13.613702    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8"
Jul 23 23:50:13.618459 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:13.618433    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8"
Jul 23 23:50:13.741000 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:13.740972    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8"
Jul 23 23:50:13.742055 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:13.742037    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8"
Jul 23 23:50:14.758366 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:14.758329    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-867102-594df9c94b-2h2cq"]
Jul 23 23:50:14.762115 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:14.762095    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-867102-594df9c94b-2h2cq"
Jul 23 23:50:14.765042 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:14.765026    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-867102-serving-cert\""
Jul 23 23:50:14.771484 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:14.771455    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-867102-594df9c94b-2h2cq"]
Jul 23 23:50:14.823675 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:14.823630    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0823b6c-309c-45de-95d6-6548497df078-maas-api-tls\") pod \"maas-api-e2e-shared-a-867102-594df9c94b-2h2cq\" (UID: \"e0823b6c-309c-45de-95d6-6548497df078\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-867102-594df9c94b-2h2cq"
Jul 23 23:50:14.823873 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:14.823754    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0823b6c-309c-45de-95d6-6548497df078-openshift-service-ca\") pod \"maas-api-e2e-shared-a-867102-594df9c94b-2h2cq\" (UID: \"e0823b6c-309c-45de-95d6-6548497df078\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-867102-594df9c94b-2h2cq"
Jul 23 23:50:14.823960 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:14.823879    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thvsp\" (UniqueName: \"kubernetes.io/projected/e0823b6c-309c-45de-95d6-6548497df078-kube-api-access-thvsp\") pod \"maas-api-e2e-shared-a-867102-594df9c94b-2h2cq\" (UID: \"e0823b6c-309c-45de-95d6-6548497df078\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-867102-594df9c94b-2h2cq"
Jul 23 23:50:14.924561 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:14.924526    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0823b6c-309c-45de-95d6-6548497df078-maas-api-tls\") pod \"maas-api-e2e-shared-a-867102-594df9c94b-2h2cq\" (UID: \"e0823b6c-309c-45de-95d6-6548497df078\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-867102-594df9c94b-2h2cq"
Jul 23 23:50:14.924760 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:14.924568    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0823b6c-309c-45de-95d6-6548497df078-openshift-service-ca\") pod \"maas-api-e2e-shared-a-867102-594df9c94b-2h2cq\" (UID: \"e0823b6c-309c-45de-95d6-6548497df078\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-867102-594df9c94b-2h2cq"
Jul 23 23:50:14.924760 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:14.924630    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thvsp\" (UniqueName: \"kubernetes.io/projected/e0823b6c-309c-45de-95d6-6548497df078-kube-api-access-thvsp\") pod \"maas-api-e2e-shared-a-867102-594df9c94b-2h2cq\" (UID: \"e0823b6c-309c-45de-95d6-6548497df078\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-867102-594df9c94b-2h2cq"
Jul 23 23:50:14.925357 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:14.925324    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0823b6c-309c-45de-95d6-6548497df078-openshift-service-ca\") pod \"maas-api-e2e-shared-a-867102-594df9c94b-2h2cq\" (UID: \"e0823b6c-309c-45de-95d6-6548497df078\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-867102-594df9c94b-2h2cq"
Jul 23 23:50:14.927122 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:14.927101    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0823b6c-309c-45de-95d6-6548497df078-maas-api-tls\") pod \"maas-api-e2e-shared-a-867102-594df9c94b-2h2cq\" (UID: \"e0823b6c-309c-45de-95d6-6548497df078\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-867102-594df9c94b-2h2cq"
Jul 23 23:50:14.935548 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:14.935511    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thvsp\" (UniqueName: \"kubernetes.io/projected/e0823b6c-309c-45de-95d6-6548497df078-kube-api-access-thvsp\") pod \"maas-api-e2e-shared-a-867102-594df9c94b-2h2cq\" (UID: \"e0823b6c-309c-45de-95d6-6548497df078\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-867102-594df9c94b-2h2cq"
Jul 23 23:50:15.075159 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:15.075068    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-867102-594df9c94b-2h2cq"
Jul 23 23:50:15.241092 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:15.241063    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-867102-594df9c94b-2h2cq"]
Jul 23 23:50:15.243007 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:50:15.242982    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0823b6c_309c_45de_95d6_6548497df078.slice/crio-3388fe03d517fe62601a04c45f24014f0986cef70b7942477e3e83618ddfe083 WatchSource:0}: Error finding container 3388fe03d517fe62601a04c45f24014f0986cef70b7942477e3e83618ddfe083: Status 404 returned error can't find the container with id 3388fe03d517fe62601a04c45f24014f0986cef70b7942477e3e83618ddfe083
Jul 23 23:50:15.244362 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:15.244344    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 23:50:15.604711 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:15.604643    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm"
Jul 23 23:50:15.733534 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:15.733500    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7434c052-65e2-43a9-9e98-d049053ed1bb-istio-podinfo\") pod \"7434c052-65e2-43a9-9e98-d049053ed1bb\" (UID: \"7434c052-65e2-43a9-9e98-d049053ed1bb\") "
Jul 23 23:50:15.733758 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:15.733548    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7434c052-65e2-43a9-9e98-d049053ed1bb-istio-data\") pod \"7434c052-65e2-43a9-9e98-d049053ed1bb\" (UID: \"7434c052-65e2-43a9-9e98-d049053ed1bb\") "
Jul 23 23:50:15.733758 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:15.733581    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7434c052-65e2-43a9-9e98-d049053ed1bb-workload-certs\") pod \"7434c052-65e2-43a9-9e98-d049053ed1bb\" (UID: \"7434c052-65e2-43a9-9e98-d049053ed1bb\") "
Jul 23 23:50:15.733758 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:15.733610    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7434c052-65e2-43a9-9e98-d049053ed1bb-credential-socket\") pod \"7434c052-65e2-43a9-9e98-d049053ed1bb\" (UID: \"7434c052-65e2-43a9-9e98-d049053ed1bb\") "
Jul 23 23:50:15.733758 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:15.733642    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7434c052-65e2-43a9-9e98-d049053ed1bb-istio-envoy\") pod \"7434c052-65e2-43a9-9e98-d049053ed1bb\" (UID: \"7434c052-65e2-43a9-9e98-d049053ed1bb\") "
Jul 23 23:50:15.733758 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:15.733723    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7434c052-65e2-43a9-9e98-d049053ed1bb-istiod-ca-cert\") pod \"7434c052-65e2-43a9-9e98-d049053ed1bb\" (UID: \"7434c052-65e2-43a9-9e98-d049053ed1bb\") "
Jul 23 23:50:15.733758 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:15.733752    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-chdvs\" (UniqueName: \"kubernetes.io/projected/7434c052-65e2-43a9-9e98-d049053ed1bb-kube-api-access-chdvs\") pod \"7434c052-65e2-43a9-9e98-d049053ed1bb\" (UID: \"7434c052-65e2-43a9-9e98-d049053ed1bb\") "
Jul 23 23:50:15.734092 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:15.733810    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7434c052-65e2-43a9-9e98-d049053ed1bb-istio-token\") pod \"7434c052-65e2-43a9-9e98-d049053ed1bb\" (UID: \"7434c052-65e2-43a9-9e98-d049053ed1bb\") "
Jul 23 23:50:15.734092 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:15.733852    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7434c052-65e2-43a9-9e98-d049053ed1bb-workload-socket\") pod \"7434c052-65e2-43a9-9e98-d049053ed1bb\" (UID: \"7434c052-65e2-43a9-9e98-d049053ed1bb\") "
Jul 23 23:50:15.734092 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:15.733946    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7434c052-65e2-43a9-9e98-d049053ed1bb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7434c052-65e2-43a9-9e98-d049053ed1bb" (UID: "7434c052-65e2-43a9-9e98-d049053ed1bb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:50:15.734092 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:15.733965    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7434c052-65e2-43a9-9e98-d049053ed1bb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7434c052-65e2-43a9-9e98-d049053ed1bb" (UID: "7434c052-65e2-43a9-9e98-d049053ed1bb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:50:15.734286 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:15.734139    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7434c052-65e2-43a9-9e98-d049053ed1bb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7434c052-65e2-43a9-9e98-d049053ed1bb" (UID: "7434c052-65e2-43a9-9e98-d049053ed1bb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:50:15.734561 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:15.734317    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7434c052-65e2-43a9-9e98-d049053ed1bb-workload-socket\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:50:15.734561 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:15.734338    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7434c052-65e2-43a9-9e98-d049053ed1bb-workload-certs\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:50:15.734561 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:15.734351    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7434c052-65e2-43a9-9e98-d049053ed1bb-credential-socket\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:50:15.734561 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:15.734394    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7434c052-65e2-43a9-9e98-d049053ed1bb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7434c052-65e2-43a9-9e98-d049053ed1bb" (UID: "7434c052-65e2-43a9-9e98-d049053ed1bb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:50:15.734805 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:15.734591    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7434c052-65e2-43a9-9e98-d049053ed1bb-istio-data" (OuterVolumeSpecName: "istio-data") pod "7434c052-65e2-43a9-9e98-d049053ed1bb" (UID: "7434c052-65e2-43a9-9e98-d049053ed1bb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:50:15.736028 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:15.735999    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7434c052-65e2-43a9-9e98-d049053ed1bb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7434c052-65e2-43a9-9e98-d049053ed1bb" (UID: "7434c052-65e2-43a9-9e98-d049053ed1bb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 23:50:15.736291 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:15.736267    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7434c052-65e2-43a9-9e98-d049053ed1bb-istio-token" (OuterVolumeSpecName: "istio-token") pod "7434c052-65e2-43a9-9e98-d049053ed1bb" (UID: "7434c052-65e2-43a9-9e98-d049053ed1bb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:50:15.736378 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:15.736305    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7434c052-65e2-43a9-9e98-d049053ed1bb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7434c052-65e2-43a9-9e98-d049053ed1bb" (UID: "7434c052-65e2-43a9-9e98-d049053ed1bb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:50:15.736473 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:15.736459    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7434c052-65e2-43a9-9e98-d049053ed1bb-kube-api-access-chdvs" (OuterVolumeSpecName: "kube-api-access-chdvs") pod "7434c052-65e2-43a9-9e98-d049053ed1bb" (UID: "7434c052-65e2-43a9-9e98-d049053ed1bb"). InnerVolumeSpecName "kube-api-access-chdvs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:50:15.752273 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:15.752245    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-867102-594df9c94b-2h2cq" event={"ID":"e0823b6c-309c-45de-95d6-6548497df078","Type":"ContainerStarted","Data":"3388fe03d517fe62601a04c45f24014f0986cef70b7942477e3e83618ddfe083"}
Jul 23 23:50:15.753835 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:15.753809    2578 generic.go:356] "Generic (PLEG): container finished" podID="7434c052-65e2-43a9-9e98-d049053ed1bb" containerID="944d0ed5d13bdde1d68bd9e20a392697378b425a2e63d2e3fb1c0b42a0ef3ca0" exitCode=0
Jul 23 23:50:15.753932 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:15.753861    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm"
Jul 23 23:50:15.753932 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:15.753895    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm" event={"ID":"7434c052-65e2-43a9-9e98-d049053ed1bb","Type":"ContainerDied","Data":"944d0ed5d13bdde1d68bd9e20a392697378b425a2e63d2e3fb1c0b42a0ef3ca0"}
Jul 23 23:50:15.754003 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:15.753948    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm" event={"ID":"7434c052-65e2-43a9-9e98-d049053ed1bb","Type":"ContainerDied","Data":"3ee287408d82b36dd932a48d6bb8706bf46d14546c3eb8097daae4a3326ba62f"}
Jul 23 23:50:15.754003 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:15.753974    2578 scope.go:117] "RemoveContainer" containerID="944d0ed5d13bdde1d68bd9e20a392697378b425a2e63d2e3fb1c0b42a0ef3ca0"
Jul 23 23:50:15.763528 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:15.763510    2578 scope.go:117] "RemoveContainer" containerID="ca4e39970d0c83d2db455c7971dae59ec75691d131c831415ea3962851d17dc0"
Jul 23 23:50:15.771671 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:15.771637    2578 scope.go:117] "RemoveContainer" containerID="944d0ed5d13bdde1d68bd9e20a392697378b425a2e63d2e3fb1c0b42a0ef3ca0"
Jul 23 23:50:15.771947 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:50:15.771927    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"944d0ed5d13bdde1d68bd9e20a392697378b425a2e63d2e3fb1c0b42a0ef3ca0\": container with ID starting with 944d0ed5d13bdde1d68bd9e20a392697378b425a2e63d2e3fb1c0b42a0ef3ca0 not found: ID does not exist" containerID="944d0ed5d13bdde1d68bd9e20a392697378b425a2e63d2e3fb1c0b42a0ef3ca0"
Jul 23 23:50:15.771997 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:15.771955    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"944d0ed5d13bdde1d68bd9e20a392697378b425a2e63d2e3fb1c0b42a0ef3ca0"} err="failed to get container status \"944d0ed5d13bdde1d68bd9e20a392697378b425a2e63d2e3fb1c0b42a0ef3ca0\": rpc error: code = NotFound desc = could not find container \"944d0ed5d13bdde1d68bd9e20a392697378b425a2e63d2e3fb1c0b42a0ef3ca0\": container with ID starting with 944d0ed5d13bdde1d68bd9e20a392697378b425a2e63d2e3fb1c0b42a0ef3ca0 not found: ID does not exist"
Jul 23 23:50:15.771997 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:15.771971    2578 scope.go:117] "RemoveContainer" containerID="ca4e39970d0c83d2db455c7971dae59ec75691d131c831415ea3962851d17dc0"
Jul 23 23:50:15.772204 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:50:15.772185    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca4e39970d0c83d2db455c7971dae59ec75691d131c831415ea3962851d17dc0\": container with ID starting with ca4e39970d0c83d2db455c7971dae59ec75691d131c831415ea3962851d17dc0 not found: ID does not exist" containerID="ca4e39970d0c83d2db455c7971dae59ec75691d131c831415ea3962851d17dc0"
Jul 23 23:50:15.772250 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:15.772209    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca4e39970d0c83d2db455c7971dae59ec75691d131c831415ea3962851d17dc0"} err="failed to get container status \"ca4e39970d0c83d2db455c7971dae59ec75691d131c831415ea3962851d17dc0\": rpc error: code = NotFound desc = could not find container \"ca4e39970d0c83d2db455c7971dae59ec75691d131c831415ea3962851d17dc0\": container with ID starting with ca4e39970d0c83d2db455c7971dae59ec75691d131c831415ea3962851d17dc0 not found: ID does not exist"
Jul 23 23:50:15.776480 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:15.776457    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm"]
Jul 23 23:50:15.781510 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:15.781490    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-9d0833-openshift-default-75fc6f8684-t8knm"]
Jul 23 23:50:15.835821 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:15.835791    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7434c052-65e2-43a9-9e98-d049053ed1bb-istio-podinfo\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:50:15.835821 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:15.835818    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7434c052-65e2-43a9-9e98-d049053ed1bb-istio-data\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:50:15.835970 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:15.835832    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7434c052-65e2-43a9-9e98-d049053ed1bb-istio-envoy\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:50:15.835970 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:15.835843    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7434c052-65e2-43a9-9e98-d049053ed1bb-istiod-ca-cert\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:50:15.835970 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:15.835855    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-chdvs\" (UniqueName: \"kubernetes.io/projected/7434c052-65e2-43a9-9e98-d049053ed1bb-kube-api-access-chdvs\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:50:15.835970 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:15.835867    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7434c052-65e2-43a9-9e98-d049053ed1bb-istio-token\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:50:16.761371 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:16.761330    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-867102-594df9c94b-2h2cq" event={"ID":"e0823b6c-309c-45de-95d6-6548497df078","Type":"ContainerStarted","Data":"6c76053c3b1ced055f25aa1ef825231205b96655b796c285398be51e5e0abe06"}
Jul 23 23:50:16.761550 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:16.761457    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-867102-594df9c94b-2h2cq"
Jul 23 23:50:16.779171 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:16.779122    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-867102-594df9c94b-2h2cq" podStartSLOduration=2.218411086 podStartE2EDuration="2.779103527s" totalImagesPullingTime="560.692441ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:50:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:50:16.776936566 +0000 UTC m=+2206.146462226" watchObservedRunningTime="2026-07-23 23:50:16.779103527 +0000 UTC m=+2206.148629190"
Jul 23 23:50:17.237392 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:17.237362    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7434c052-65e2-43a9-9e98-d049053ed1bb" path="/var/lib/kubelet/pods/7434c052-65e2-43a9-9e98-d049053ed1bb/volumes"
Jul 23 23:50:19.824444 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:19.824403    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh"]
Jul 23 23:50:19.824852 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:19.824799    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7434c052-65e2-43a9-9e98-d049053ed1bb" containerName="istio-proxy"
Jul 23 23:50:19.824852 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:19.824814    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7434c052-65e2-43a9-9e98-d049053ed1bb" containerName="istio-proxy"
Jul 23 23:50:19.824852 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:19.824828    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7434c052-65e2-43a9-9e98-d049053ed1bb" containerName="istio-proxy"
Jul 23 23:50:19.824852 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:19.824834    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7434c052-65e2-43a9-9e98-d049053ed1bb" containerName="istio-proxy"
Jul 23 23:50:19.825017 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:19.824894    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7434c052-65e2-43a9-9e98-d049053ed1bb" containerName="istio-proxy"
Jul 23 23:50:19.828474 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:19.828452    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh"
Jul 23 23:50:19.831227 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:19.831197    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-fc333b-openshift-default-dockercfg-jjlj6\""
Jul 23 23:50:19.842096 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:19.842074    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh"]
Jul 23 23:50:19.878586 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:19.878549    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8fbe86f-2de9-44ef-861e-70362bd1476e-istio-podinfo\") pod \"e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh\" (UID: \"e8fbe86f-2de9-44ef-861e-70362bd1476e\") " pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh"
Jul 23 23:50:19.878586 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:19.878592    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8fbe86f-2de9-44ef-861e-70362bd1476e-istio-token\") pod \"e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh\" (UID: \"e8fbe86f-2de9-44ef-861e-70362bd1476e\") " pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh"
Jul 23 23:50:19.878846 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:19.878613    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8fbe86f-2de9-44ef-861e-70362bd1476e-credential-socket\") pod \"e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh\" (UID: \"e8fbe86f-2de9-44ef-861e-70362bd1476e\") " pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh"
Jul 23 23:50:19.878846 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:19.878680    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8fbe86f-2de9-44ef-861e-70362bd1476e-istio-envoy\") pod \"e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh\" (UID: \"e8fbe86f-2de9-44ef-861e-70362bd1476e\") " pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh"
Jul 23 23:50:19.878846 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:19.878749    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8fbe86f-2de9-44ef-861e-70362bd1476e-istiod-ca-cert\") pod \"e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh\" (UID: \"e8fbe86f-2de9-44ef-861e-70362bd1476e\") " pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh"
Jul 23 23:50:19.878846 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:19.878789    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8fbe86f-2de9-44ef-861e-70362bd1476e-workload-certs\") pod \"e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh\" (UID: \"e8fbe86f-2de9-44ef-861e-70362bd1476e\") " pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh"
Jul 23 23:50:19.878846 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:19.878818    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8fbe86f-2de9-44ef-861e-70362bd1476e-workload-socket\") pod \"e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh\" (UID: \"e8fbe86f-2de9-44ef-861e-70362bd1476e\") " pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh"
Jul 23 23:50:19.879041 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:19.878891    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8fbe86f-2de9-44ef-861e-70362bd1476e-istio-data\") pod \"e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh\" (UID: \"e8fbe86f-2de9-44ef-861e-70362bd1476e\") " pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh"
Jul 23 23:50:19.879041 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:19.878922    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czxkc\" (UniqueName: \"kubernetes.io/projected/e8fbe86f-2de9-44ef-861e-70362bd1476e-kube-api-access-czxkc\") pod \"e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh\" (UID: \"e8fbe86f-2de9-44ef-861e-70362bd1476e\") " pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh"
Jul 23 23:50:19.980146 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:19.980108    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8fbe86f-2de9-44ef-861e-70362bd1476e-istio-data\") pod \"e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh\" (UID: \"e8fbe86f-2de9-44ef-861e-70362bd1476e\") " pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh"
Jul 23 23:50:19.980331 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:19.980155    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czxkc\" (UniqueName: \"kubernetes.io/projected/e8fbe86f-2de9-44ef-861e-70362bd1476e-kube-api-access-czxkc\") pod \"e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh\" (UID: \"e8fbe86f-2de9-44ef-861e-70362bd1476e\") " pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh"
Jul 23 23:50:19.980331 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:19.980208    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8fbe86f-2de9-44ef-861e-70362bd1476e-istio-podinfo\") pod \"e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh\" (UID: \"e8fbe86f-2de9-44ef-861e-70362bd1476e\") " pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh"
Jul 23 23:50:19.980331 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:19.980245    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8fbe86f-2de9-44ef-861e-70362bd1476e-istio-token\") pod \"e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh\" (UID: \"e8fbe86f-2de9-44ef-861e-70362bd1476e\") " pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh"
Jul 23 23:50:19.980453 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:19.980427    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8fbe86f-2de9-44ef-861e-70362bd1476e-credential-socket\") pod \"e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh\" (UID: \"e8fbe86f-2de9-44ef-861e-70362bd1476e\") " pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh"
Jul 23 23:50:19.980498 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:19.980468    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8fbe86f-2de9-44ef-861e-70362bd1476e-istio-envoy\") pod \"e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh\" (UID: \"e8fbe86f-2de9-44ef-861e-70362bd1476e\") " pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh"
Jul 23 23:50:19.980540 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:19.980510    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8fbe86f-2de9-44ef-861e-70362bd1476e-istiod-ca-cert\") pod \"e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh\" (UID: \"e8fbe86f-2de9-44ef-861e-70362bd1476e\") " pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh"
Jul 23 23:50:19.980589 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:19.980542    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8fbe86f-2de9-44ef-861e-70362bd1476e-workload-certs\") pod \"e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh\" (UID: \"e8fbe86f-2de9-44ef-861e-70362bd1476e\") " pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh"
Jul 23 23:50:19.980589 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:19.980549    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8fbe86f-2de9-44ef-861e-70362bd1476e-istio-data\") pod \"e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh\" (UID: \"e8fbe86f-2de9-44ef-861e-70362bd1476e\") " pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh"
Jul 23 23:50:19.980589 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:19.980582    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8fbe86f-2de9-44ef-861e-70362bd1476e-workload-socket\") pod \"e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh\" (UID: \"e8fbe86f-2de9-44ef-861e-70362bd1476e\") " pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh"
Jul 23 23:50:19.980812 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:19.980787    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8fbe86f-2de9-44ef-861e-70362bd1476e-credential-socket\") pod \"e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh\" (UID: \"e8fbe86f-2de9-44ef-861e-70362bd1476e\") " pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh"
Jul 23 23:50:19.980887 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:19.980831    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8fbe86f-2de9-44ef-861e-70362bd1476e-workload-certs\") pod \"e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh\" (UID: \"e8fbe86f-2de9-44ef-861e-70362bd1476e\") " pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh"
Jul 23 23:50:19.980987 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:19.980945    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8fbe86f-2de9-44ef-861e-70362bd1476e-workload-socket\") pod \"e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh\" (UID: \"e8fbe86f-2de9-44ef-861e-70362bd1476e\") " pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh"
Jul 23 23:50:19.981187 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:19.981133    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8fbe86f-2de9-44ef-861e-70362bd1476e-istiod-ca-cert\") pod \"e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh\" (UID: \"e8fbe86f-2de9-44ef-861e-70362bd1476e\") " pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh"
Jul 23 23:50:19.982834 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:19.982805    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8fbe86f-2de9-44ef-861e-70362bd1476e-istio-envoy\") pod \"e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh\" (UID: \"e8fbe86f-2de9-44ef-861e-70362bd1476e\") " pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh"
Jul 23 23:50:19.982834 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:19.982827    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8fbe86f-2de9-44ef-861e-70362bd1476e-istio-podinfo\") pod \"e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh\" (UID: \"e8fbe86f-2de9-44ef-861e-70362bd1476e\") " pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh"
Jul 23 23:50:19.987947 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:19.987923    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8fbe86f-2de9-44ef-861e-70362bd1476e-istio-token\") pod \"e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh\" (UID: \"e8fbe86f-2de9-44ef-861e-70362bd1476e\") " pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh"
Jul 23 23:50:19.988055 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:19.988045    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czxkc\" (UniqueName: \"kubernetes.io/projected/e8fbe86f-2de9-44ef-861e-70362bd1476e-kube-api-access-czxkc\") pod \"e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh\" (UID: \"e8fbe86f-2de9-44ef-861e-70362bd1476e\") " pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh"
Jul 23 23:50:20.139129 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:20.139032    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh"
Jul 23 23:50:20.281043 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:20.281015    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh"]
Jul 23 23:50:20.283696 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:50:20.283668    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8fbe86f_2de9_44ef_861e_70362bd1476e.slice/crio-148030f15a85bc921587c7f2ab15ce86e87e0b35035b02a5e51e6948b1ef7553 WatchSource:0}: Error finding container 148030f15a85bc921587c7f2ab15ce86e87e0b35035b02a5e51e6948b1ef7553: Status 404 returned error can't find the container with id 148030f15a85bc921587c7f2ab15ce86e87e0b35035b02a5e51e6948b1ef7553
Jul 23 23:50:20.285819 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:20.285790    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:50:20.285902 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:20.285851    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:50:20.285902 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:20.285881    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:50:20.782220 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:20.782182    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh" event={"ID":"e8fbe86f-2de9-44ef-861e-70362bd1476e","Type":"ContainerStarted","Data":"07c977f76601de18ecfe7ef41c4c4a96f4b0556eb049a70efbc9697c18c86f2d"}
Jul 23 23:50:20.782408 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:20.782244    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh" event={"ID":"e8fbe86f-2de9-44ef-861e-70362bd1476e","Type":"ContainerStarted","Data":"148030f15a85bc921587c7f2ab15ce86e87e0b35035b02a5e51e6948b1ef7553"}
Jul 23 23:50:20.801709 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:20.801630    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh" podStartSLOduration=1.801607791 podStartE2EDuration="1.801607791s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:50:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:50:20.800841133 +0000 UTC m=+2210.170366793" watchObservedRunningTime="2026-07-23 23:50:20.801607791 +0000 UTC m=+2210.171133452"
Jul 23 23:50:21.139779 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:21.139683    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh"
Jul 23 23:50:21.145505 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:21.145478    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh"
Jul 23 23:50:21.791395 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:21.791359    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh"
Jul 23 23:50:21.795466 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:21.795436    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh"
Jul 23 23:50:22.771687 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:22.771644    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-867102-594df9c94b-2h2cq"
Jul 23 23:50:22.791020 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:22.790987    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-fc333b-589fb55cb5-qjl47"]
Jul 23 23:50:22.791850 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:22.791825    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7434c052-65e2-43a9-9e98-d049053ed1bb" containerName="istio-proxy"
Jul 23 23:50:22.796088 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:22.796060    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fc333b-589fb55cb5-qjl47"
Jul 23 23:50:22.798984 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:22.798958    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-fc333b-serving-cert\""
Jul 23 23:50:22.808601 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:22.808578    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-fc333b-589fb55cb5-qjl47"]
Jul 23 23:50:22.911449 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:22.911407    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpd86\" (UniqueName: \"kubernetes.io/projected/0ea0f5cd-26af-474a-b36a-73b1ddb2c501-kube-api-access-zpd86\") pod \"maas-api-e2e-shared-b-fc333b-589fb55cb5-qjl47\" (UID: \"0ea0f5cd-26af-474a-b36a-73b1ddb2c501\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fc333b-589fb55cb5-qjl47"
Jul 23 23:50:22.911688 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:22.911510    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ea0f5cd-26af-474a-b36a-73b1ddb2c501-openshift-service-ca\") pod \"maas-api-e2e-shared-b-fc333b-589fb55cb5-qjl47\" (UID: \"0ea0f5cd-26af-474a-b36a-73b1ddb2c501\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fc333b-589fb55cb5-qjl47"
Jul 23 23:50:22.911775 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:22.911689    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ea0f5cd-26af-474a-b36a-73b1ddb2c501-maas-api-tls\") pod \"maas-api-e2e-shared-b-fc333b-589fb55cb5-qjl47\" (UID: \"0ea0f5cd-26af-474a-b36a-73b1ddb2c501\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fc333b-589fb55cb5-qjl47"
Jul 23 23:50:23.012542 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:23.012504    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ea0f5cd-26af-474a-b36a-73b1ddb2c501-maas-api-tls\") pod \"maas-api-e2e-shared-b-fc333b-589fb55cb5-qjl47\" (UID: \"0ea0f5cd-26af-474a-b36a-73b1ddb2c501\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fc333b-589fb55cb5-qjl47"
Jul 23 23:50:23.012776 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:23.012577    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpd86\" (UniqueName: \"kubernetes.io/projected/0ea0f5cd-26af-474a-b36a-73b1ddb2c501-kube-api-access-zpd86\") pod \"maas-api-e2e-shared-b-fc333b-589fb55cb5-qjl47\" (UID: \"0ea0f5cd-26af-474a-b36a-73b1ddb2c501\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fc333b-589fb55cb5-qjl47"
Jul 23 23:50:23.012776 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:23.012685    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ea0f5cd-26af-474a-b36a-73b1ddb2c501-openshift-service-ca\") pod \"maas-api-e2e-shared-b-fc333b-589fb55cb5-qjl47\" (UID: \"0ea0f5cd-26af-474a-b36a-73b1ddb2c501\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fc333b-589fb55cb5-qjl47"
Jul 23 23:50:23.012776 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:50:23.012705    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-fc333b-serving-cert: secret "maas-api-e2e-shared-b-fc333b-serving-cert" not found
Jul 23 23:50:23.012776 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:50:23.012775    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ea0f5cd-26af-474a-b36a-73b1ddb2c501-maas-api-tls podName:0ea0f5cd-26af-474a-b36a-73b1ddb2c501 nodeName:}" failed. No retries permitted until 2026-07-23 23:50:23.512753506 +0000 UTC m=+2212.882279146 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0ea0f5cd-26af-474a-b36a-73b1ddb2c501-maas-api-tls") pod "maas-api-e2e-shared-b-fc333b-589fb55cb5-qjl47" (UID: "0ea0f5cd-26af-474a-b36a-73b1ddb2c501") : secret "maas-api-e2e-shared-b-fc333b-serving-cert" not found
Jul 23 23:50:23.013333 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:23.013314    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ea0f5cd-26af-474a-b36a-73b1ddb2c501-openshift-service-ca\") pod \"maas-api-e2e-shared-b-fc333b-589fb55cb5-qjl47\" (UID: \"0ea0f5cd-26af-474a-b36a-73b1ddb2c501\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fc333b-589fb55cb5-qjl47"
Jul 23 23:50:23.021995 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:23.021939    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpd86\" (UniqueName: \"kubernetes.io/projected/0ea0f5cd-26af-474a-b36a-73b1ddb2c501-kube-api-access-zpd86\") pod \"maas-api-e2e-shared-b-fc333b-589fb55cb5-qjl47\" (UID: \"0ea0f5cd-26af-474a-b36a-73b1ddb2c501\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fc333b-589fb55cb5-qjl47"
Jul 23 23:50:23.518206 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:23.518163    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ea0f5cd-26af-474a-b36a-73b1ddb2c501-maas-api-tls\") pod \"maas-api-e2e-shared-b-fc333b-589fb55cb5-qjl47\" (UID: \"0ea0f5cd-26af-474a-b36a-73b1ddb2c501\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fc333b-589fb55cb5-qjl47"
Jul 23 23:50:23.520832 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:23.520814    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ea0f5cd-26af-474a-b36a-73b1ddb2c501-maas-api-tls\") pod \"maas-api-e2e-shared-b-fc333b-589fb55cb5-qjl47\" (UID: \"0ea0f5cd-26af-474a-b36a-73b1ddb2c501\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fc333b-589fb55cb5-qjl47"
Jul 23 23:50:23.712624 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:23.712585    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fc333b-589fb55cb5-qjl47"
Jul 23 23:50:23.889358 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:23.889315    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-fc333b-589fb55cb5-qjl47"]
Jul 23 23:50:23.897600 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:50:23.897564    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ea0f5cd_26af_474a_b36a_73b1ddb2c501.slice/crio-d63893b82f6b505e78d1f01b7000c85ec40e0169a9217d3973d6238f43e53a04 WatchSource:0}: Error finding container d63893b82f6b505e78d1f01b7000c85ec40e0169a9217d3973d6238f43e53a04: Status 404 returned error can't find the container with id d63893b82f6b505e78d1f01b7000c85ec40e0169a9217d3973d6238f43e53a04
Jul 23 23:50:24.813236 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:24.813198    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fc333b-589fb55cb5-qjl47" event={"ID":"0ea0f5cd-26af-474a-b36a-73b1ddb2c501","Type":"ContainerStarted","Data":"5dd47cee09b3d311732321ba939a46d4c2c5beb3f88403b8c0bb702a2d5eb599"}
Jul 23 23:50:24.813416 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:24.813251    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fc333b-589fb55cb5-qjl47" event={"ID":"0ea0f5cd-26af-474a-b36a-73b1ddb2c501","Type":"ContainerStarted","Data":"d63893b82f6b505e78d1f01b7000c85ec40e0169a9217d3973d6238f43e53a04"}
Jul 23 23:50:24.813416 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:24.813279    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fc333b-589fb55cb5-qjl47"
Jul 23 23:50:24.835543 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:24.835492    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fc333b-589fb55cb5-qjl47" podStartSLOduration=2.43795783 podStartE2EDuration="2.835476392s" totalImagesPullingTime="397.518562ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 23:50:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:50:24.833589612 +0000 UTC m=+2214.203115273" watchObservedRunningTime="2026-07-23 23:50:24.835476392 +0000 UTC m=+2214.205002050"
Jul 23 23:50:29.782913 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:29.782857    2578 patch_prober.go:28] interesting pod/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 23 23:50:29.783460 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:29.782947    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8" podUID="13b2835b-0e85-44a3-a4b1-9172fff9f137" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 23 23:50:30.823640 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:30.823611    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fc333b-589fb55cb5-qjl47"
Jul 23 23:50:30.845861 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:30.845813    2578 generic.go:356] "Generic (PLEG): container finished" podID="13b2835b-0e85-44a3-a4b1-9172fff9f137" containerID="92108de625269ac67a2674374b54b817f3c8e6821bac437554ac7dc168ad52a7" exitCode=0
Jul 23 23:50:30.846201 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:30.845897    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8" event={"ID":"13b2835b-0e85-44a3-a4b1-9172fff9f137","Type":"ContainerDied","Data":"92108de625269ac67a2674374b54b817f3c8e6821bac437554ac7dc168ad52a7"}
Jul 23 23:50:30.846361 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:30.846343    2578 scope.go:117] "RemoveContainer" containerID="92108de625269ac67a2674374b54b817f3c8e6821bac437554ac7dc168ad52a7"
Jul 23 23:50:30.847674 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:30.847616    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:50:30.847773 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:30.847732    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:50:30.847813 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:30.847782    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:50:31.853142 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:31.853101    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8" event={"ID":"13b2835b-0e85-44a3-a4b1-9172fff9f137","Type":"ContainerStarted","Data":"bc76ce0ad24ebf12efa8268f9ba72d035f338f587ca4dbea109328cea036bb71"}
Jul 23 23:50:32.613640 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:32.613590    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8"
Jul 23 23:50:32.613900 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:32.613763    2578 patch_prober.go:28] interesting pod/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 23 23:50:32.613900 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:32.613817    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8" podUID="13b2835b-0e85-44a3-a4b1-9172fff9f137" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 23 23:50:33.614261 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:33.614213    2578 patch_prober.go:28] interesting pod/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 23 23:50:33.614631 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:33.614291    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8" podUID="13b2835b-0e85-44a3-a4b1-9172fff9f137" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 23 23:50:34.614330 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:34.614288    2578 patch_prober.go:28] interesting pod/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 23 23:50:34.614867 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:34.614348    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8" podUID="13b2835b-0e85-44a3-a4b1-9172fff9f137" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 23 23:50:35.614287 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:35.614242    2578 patch_prober.go:28] interesting pod/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 23 23:50:35.614471 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:35.614323    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8" podUID="13b2835b-0e85-44a3-a4b1-9172fff9f137" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 23 23:50:36.617845 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:36.617815    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8"
Jul 23 23:50:36.618271 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:36.618197    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8"
Jul 23 23:50:36.618808 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:36.618791    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8"
Jul 23 23:50:36.793207 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:36.793180    2578 patch_prober.go:28] interesting pod/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 23 23:50:36.793344 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:36.793232    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh" podUID="e8fbe86f-2de9-44ef-861e-70362bd1476e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 23 23:50:36.885707 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:36.885600    2578 generic.go:356] "Generic (PLEG): container finished" podID="e8fbe86f-2de9-44ef-861e-70362bd1476e" containerID="07c977f76601de18ecfe7ef41c4c4a96f4b0556eb049a70efbc9697c18c86f2d" exitCode=0
Jul 23 23:50:36.885707 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:36.885680    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh" event={"ID":"e8fbe86f-2de9-44ef-861e-70362bd1476e","Type":"ContainerDied","Data":"07c977f76601de18ecfe7ef41c4c4a96f4b0556eb049a70efbc9697c18c86f2d"}
Jul 23 23:50:36.886192 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:36.886171    2578 scope.go:117] "RemoveContainer" containerID="07c977f76601de18ecfe7ef41c4c4a96f4b0556eb049a70efbc9697c18c86f2d"
Jul 23 23:50:36.887194 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:36.887167    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:50:36.887321 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:36.887257    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:50:36.887321 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:36.887306    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 23:50:37.896072 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:37.896034    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh" event={"ID":"e8fbe86f-2de9-44ef-861e-70362bd1476e","Type":"ContainerStarted","Data":"45fc908f232f42f8e38c4b48af73829544b69b600beb0c5e3f8e88dba6b0ebd0"}
Jul 23 23:50:38.139845 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:38.139802    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh"
Jul 23 23:50:38.140105 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:38.140065    2578 patch_prober.go:28] interesting pod/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 23 23:50:38.140209 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:38.140124    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh" podUID="e8fbe86f-2de9-44ef-861e-70362bd1476e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 23 23:50:39.140097 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:39.140057    2578 patch_prober.go:28] interesting pod/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 23 23:50:39.140495 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:39.140120    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh" podUID="e8fbe86f-2de9-44ef-861e-70362bd1476e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 23 23:50:40.139797 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:40.139759    2578 patch_prober.go:28] interesting pod/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 23 23:50:40.139981 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:40.139814    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh" podUID="e8fbe86f-2de9-44ef-861e-70362bd1476e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 23 23:50:41.140146 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:41.140100    2578 patch_prober.go:28] interesting pod/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 23 23:50:41.140617 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:41.140171    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh" podUID="e8fbe86f-2de9-44ef-861e-70362bd1476e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 23 23:50:42.144052 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:42.144016    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh"
Jul 23 23:50:42.144514 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:42.144490    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh"
Jul 23 23:50:42.145151 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:50:42.145134    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh"
Jul 23 23:56:37.843519 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:37.843478    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-867102-594df9c94b-2h2cq"]
Jul 23 23:56:37.844087 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:37.843750    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-867102-594df9c94b-2h2cq" podUID="e0823b6c-309c-45de-95d6-6548497df078" containerName="maas-api" containerID="cri-o://6c76053c3b1ced055f25aa1ef825231205b96655b796c285398be51e5e0abe06" gracePeriod=30
Jul 23 23:56:38.107174 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:38.107110    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-867102-594df9c94b-2h2cq"
Jul 23 23:56:38.188377 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:38.188333    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0823b6c-309c-45de-95d6-6548497df078-maas-api-tls\") pod \"e0823b6c-309c-45de-95d6-6548497df078\" (UID: \"e0823b6c-309c-45de-95d6-6548497df078\") "
Jul 23 23:56:38.188377 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:38.188379    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-thvsp\" (UniqueName: \"kubernetes.io/projected/e0823b6c-309c-45de-95d6-6548497df078-kube-api-access-thvsp\") pod \"e0823b6c-309c-45de-95d6-6548497df078\" (UID: \"e0823b6c-309c-45de-95d6-6548497df078\") "
Jul 23 23:56:38.188595 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:38.188412    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0823b6c-309c-45de-95d6-6548497df078-openshift-service-ca\") pod \"e0823b6c-309c-45de-95d6-6548497df078\" (UID: \"e0823b6c-309c-45de-95d6-6548497df078\") "
Jul 23 23:56:38.188876 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:38.188845    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0823b6c-309c-45de-95d6-6548497df078-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e0823b6c-309c-45de-95d6-6548497df078" (UID: "e0823b6c-309c-45de-95d6-6548497df078"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:56:38.190711 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:38.190623    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0823b6c-309c-45de-95d6-6548497df078-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e0823b6c-309c-45de-95d6-6548497df078" (UID: "e0823b6c-309c-45de-95d6-6548497df078"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 23:56:38.190711 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:38.190638    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0823b6c-309c-45de-95d6-6548497df078-kube-api-access-thvsp" (OuterVolumeSpecName: "kube-api-access-thvsp") pod "e0823b6c-309c-45de-95d6-6548497df078" (UID: "e0823b6c-309c-45de-95d6-6548497df078"). InnerVolumeSpecName "kube-api-access-thvsp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:56:38.290051 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:38.290011    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0823b6c-309c-45de-95d6-6548497df078-maas-api-tls\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:56:38.290051 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:38.290047    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-thvsp\" (UniqueName: \"kubernetes.io/projected/e0823b6c-309c-45de-95d6-6548497df078-kube-api-access-thvsp\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:56:38.290316 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:38.290061    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0823b6c-309c-45de-95d6-6548497df078-openshift-service-ca\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:56:38.672884 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:38.672847    2578 generic.go:356] "Generic (PLEG): container finished" podID="e0823b6c-309c-45de-95d6-6548497df078" containerID="6c76053c3b1ced055f25aa1ef825231205b96655b796c285398be51e5e0abe06" exitCode=0
Jul 23 23:56:38.673055 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:38.672894    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-867102-594df9c94b-2h2cq"
Jul 23 23:56:38.673055 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:38.672930    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-867102-594df9c94b-2h2cq" event={"ID":"e0823b6c-309c-45de-95d6-6548497df078","Type":"ContainerDied","Data":"6c76053c3b1ced055f25aa1ef825231205b96655b796c285398be51e5e0abe06"}
Jul 23 23:56:38.673055 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:38.672986    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-867102-594df9c94b-2h2cq" event={"ID":"e0823b6c-309c-45de-95d6-6548497df078","Type":"ContainerDied","Data":"3388fe03d517fe62601a04c45f24014f0986cef70b7942477e3e83618ddfe083"}
Jul 23 23:56:38.673055 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:38.673009    2578 scope.go:117] "RemoveContainer" containerID="6c76053c3b1ced055f25aa1ef825231205b96655b796c285398be51e5e0abe06"
Jul 23 23:56:38.682856 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:38.682838    2578 scope.go:117] "RemoveContainer" containerID="6c76053c3b1ced055f25aa1ef825231205b96655b796c285398be51e5e0abe06"
Jul 23 23:56:38.683144 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:56:38.683127    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c76053c3b1ced055f25aa1ef825231205b96655b796c285398be51e5e0abe06\": container with ID starting with 6c76053c3b1ced055f25aa1ef825231205b96655b796c285398be51e5e0abe06 not found: ID does not exist" containerID="6c76053c3b1ced055f25aa1ef825231205b96655b796c285398be51e5e0abe06"
Jul 23 23:56:38.683185 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:38.683159    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c76053c3b1ced055f25aa1ef825231205b96655b796c285398be51e5e0abe06"} err="failed to get container status \"6c76053c3b1ced055f25aa1ef825231205b96655b796c285398be51e5e0abe06\": rpc error: code = NotFound desc = could not find container \"6c76053c3b1ced055f25aa1ef825231205b96655b796c285398be51e5e0abe06\": container with ID starting with 6c76053c3b1ced055f25aa1ef825231205b96655b796c285398be51e5e0abe06 not found: ID does not exist"
Jul 23 23:56:38.694774 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:38.694744    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-867102-594df9c94b-2h2cq"]
Jul 23 23:56:38.698846 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:38.698818    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-867102-594df9c94b-2h2cq"]
Jul 23 23:56:39.236910 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:39.236871    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0823b6c-309c-45de-95d6-6548497df078" path="/var/lib/kubelet/pods/e0823b6c-309c-45de-95d6-6548497df078/volumes"
Jul 23 23:56:48.635894 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:48.635856    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8"]
Jul 23 23:56:48.636336 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:48.636129    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8" podUID="13b2835b-0e85-44a3-a4b1-9172fff9f137" containerName="istio-proxy" containerID="cri-o://bc76ce0ad24ebf12efa8268f9ba72d035f338f587ca4dbea109328cea036bb71" gracePeriod=30
Jul 23 23:56:51.620446 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:51.620372    2578 patch_prober.go:28] interesting pod/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 23 23:56:51.620446 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:51.620435    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8" podUID="13b2835b-0e85-44a3-a4b1-9172fff9f137" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 23 23:56:53.889304 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:53.889281    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8"
Jul 23 23:56:54.051883 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:54.051847    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6srdt\" (UniqueName: \"kubernetes.io/projected/13b2835b-0e85-44a3-a4b1-9172fff9f137-kube-api-access-6srdt\") pod \"13b2835b-0e85-44a3-a4b1-9172fff9f137\" (UID: \"13b2835b-0e85-44a3-a4b1-9172fff9f137\") "
Jul 23 23:56:54.052145 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:54.051900    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13b2835b-0e85-44a3-a4b1-9172fff9f137-workload-socket\") pod \"13b2835b-0e85-44a3-a4b1-9172fff9f137\" (UID: \"13b2835b-0e85-44a3-a4b1-9172fff9f137\") "
Jul 23 23:56:54.052145 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:54.051921    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13b2835b-0e85-44a3-a4b1-9172fff9f137-istio-envoy\") pod \"13b2835b-0e85-44a3-a4b1-9172fff9f137\" (UID: \"13b2835b-0e85-44a3-a4b1-9172fff9f137\") "
Jul 23 23:56:54.052145 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:54.051939    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13b2835b-0e85-44a3-a4b1-9172fff9f137-workload-certs\") pod \"13b2835b-0e85-44a3-a4b1-9172fff9f137\" (UID: \"13b2835b-0e85-44a3-a4b1-9172fff9f137\") "
Jul 23 23:56:54.052145 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:54.051960    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13b2835b-0e85-44a3-a4b1-9172fff9f137-credential-socket\") pod \"13b2835b-0e85-44a3-a4b1-9172fff9f137\" (UID: \"13b2835b-0e85-44a3-a4b1-9172fff9f137\") "
Jul 23 23:56:54.052145 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:54.052003    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13b2835b-0e85-44a3-a4b1-9172fff9f137-istio-data\") pod \"13b2835b-0e85-44a3-a4b1-9172fff9f137\" (UID: \"13b2835b-0e85-44a3-a4b1-9172fff9f137\") "
Jul 23 23:56:54.052145 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:54.052057    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13b2835b-0e85-44a3-a4b1-9172fff9f137-istiod-ca-cert\") pod \"13b2835b-0e85-44a3-a4b1-9172fff9f137\" (UID: \"13b2835b-0e85-44a3-a4b1-9172fff9f137\") "
Jul 23 23:56:54.052145 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:54.052081    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13b2835b-0e85-44a3-a4b1-9172fff9f137-istio-token\") pod \"13b2835b-0e85-44a3-a4b1-9172fff9f137\" (UID: \"13b2835b-0e85-44a3-a4b1-9172fff9f137\") "
Jul 23 23:56:54.052145 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:54.052113    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13b2835b-0e85-44a3-a4b1-9172fff9f137-istio-podinfo\") pod \"13b2835b-0e85-44a3-a4b1-9172fff9f137\" (UID: \"13b2835b-0e85-44a3-a4b1-9172fff9f137\") "
Jul 23 23:56:54.052583 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:54.052187    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13b2835b-0e85-44a3-a4b1-9172fff9f137-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "13b2835b-0e85-44a3-a4b1-9172fff9f137" (UID: "13b2835b-0e85-44a3-a4b1-9172fff9f137"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:56:54.052583 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:54.052390    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13b2835b-0e85-44a3-a4b1-9172fff9f137-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "13b2835b-0e85-44a3-a4b1-9172fff9f137" (UID: "13b2835b-0e85-44a3-a4b1-9172fff9f137"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:56:54.052583 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:54.052565    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13b2835b-0e85-44a3-a4b1-9172fff9f137-workload-socket\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:56:54.052950 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:54.052585    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13b2835b-0e85-44a3-a4b1-9172fff9f137-workload-certs\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:56:54.052950 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:54.052594    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13b2835b-0e85-44a3-a4b1-9172fff9f137-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "13b2835b-0e85-44a3-a4b1-9172fff9f137" (UID: "13b2835b-0e85-44a3-a4b1-9172fff9f137"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:56:54.052950 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:54.052689    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13b2835b-0e85-44a3-a4b1-9172fff9f137-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "13b2835b-0e85-44a3-a4b1-9172fff9f137" (UID: "13b2835b-0e85-44a3-a4b1-9172fff9f137"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:56:54.053283 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:54.053251    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13b2835b-0e85-44a3-a4b1-9172fff9f137-istio-data" (OuterVolumeSpecName: "istio-data") pod "13b2835b-0e85-44a3-a4b1-9172fff9f137" (UID: "13b2835b-0e85-44a3-a4b1-9172fff9f137"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:56:54.054419 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:54.054392    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13b2835b-0e85-44a3-a4b1-9172fff9f137-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "13b2835b-0e85-44a3-a4b1-9172fff9f137" (UID: "13b2835b-0e85-44a3-a4b1-9172fff9f137"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:56:54.054419 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:54.054408    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13b2835b-0e85-44a3-a4b1-9172fff9f137-kube-api-access-6srdt" (OuterVolumeSpecName: "kube-api-access-6srdt") pod "13b2835b-0e85-44a3-a4b1-9172fff9f137" (UID: "13b2835b-0e85-44a3-a4b1-9172fff9f137"). InnerVolumeSpecName "kube-api-access-6srdt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:56:54.054555 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:54.054475    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/13b2835b-0e85-44a3-a4b1-9172fff9f137-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "13b2835b-0e85-44a3-a4b1-9172fff9f137" (UID: "13b2835b-0e85-44a3-a4b1-9172fff9f137"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 23:56:54.054555 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:54.054491    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13b2835b-0e85-44a3-a4b1-9172fff9f137-istio-token" (OuterVolumeSpecName: "istio-token") pod "13b2835b-0e85-44a3-a4b1-9172fff9f137" (UID: "13b2835b-0e85-44a3-a4b1-9172fff9f137"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:56:54.154086 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:54.154054    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6srdt\" (UniqueName: \"kubernetes.io/projected/13b2835b-0e85-44a3-a4b1-9172fff9f137-kube-api-access-6srdt\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:56:54.154086 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:54.154080    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13b2835b-0e85-44a3-a4b1-9172fff9f137-istio-envoy\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:56:54.154086 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:54.154089    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13b2835b-0e85-44a3-a4b1-9172fff9f137-credential-socket\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:56:54.154307 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:54.154099    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13b2835b-0e85-44a3-a4b1-9172fff9f137-istio-data\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:56:54.154307 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:54.154108    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13b2835b-0e85-44a3-a4b1-9172fff9f137-istiod-ca-cert\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:56:54.154307 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:54.154116    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13b2835b-0e85-44a3-a4b1-9172fff9f137-istio-token\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:56:54.154307 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:54.154124    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13b2835b-0e85-44a3-a4b1-9172fff9f137-istio-podinfo\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:56:54.755891 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:54.755854    2578 generic.go:356] "Generic (PLEG): container finished" podID="13b2835b-0e85-44a3-a4b1-9172fff9f137" containerID="bc76ce0ad24ebf12efa8268f9ba72d035f338f587ca4dbea109328cea036bb71" exitCode=0
Jul 23 23:56:54.756082 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:54.755915    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8"
Jul 23 23:56:54.756082 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:54.755945    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8" event={"ID":"13b2835b-0e85-44a3-a4b1-9172fff9f137","Type":"ContainerDied","Data":"bc76ce0ad24ebf12efa8268f9ba72d035f338f587ca4dbea109328cea036bb71"}
Jul 23 23:56:54.756082 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:54.756026    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8" event={"ID":"13b2835b-0e85-44a3-a4b1-9172fff9f137","Type":"ContainerDied","Data":"581d63b11c0ba7bd2e4aefa149d2c789fb375c71f1a73ce0d05c9cbb207eb658"}
Jul 23 23:56:54.756082 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:54.756048    2578 scope.go:117] "RemoveContainer" containerID="bc76ce0ad24ebf12efa8268f9ba72d035f338f587ca4dbea109328cea036bb71"
Jul 23 23:56:54.767199 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:54.767179    2578 scope.go:117] "RemoveContainer" containerID="92108de625269ac67a2674374b54b817f3c8e6821bac437554ac7dc168ad52a7"
Jul 23 23:56:54.775193 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:54.775174    2578 scope.go:117] "RemoveContainer" containerID="bc76ce0ad24ebf12efa8268f9ba72d035f338f587ca4dbea109328cea036bb71"
Jul 23 23:56:54.775440 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:56:54.775417    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc76ce0ad24ebf12efa8268f9ba72d035f338f587ca4dbea109328cea036bb71\": container with ID starting with bc76ce0ad24ebf12efa8268f9ba72d035f338f587ca4dbea109328cea036bb71 not found: ID does not exist" containerID="bc76ce0ad24ebf12efa8268f9ba72d035f338f587ca4dbea109328cea036bb71"
Jul 23 23:56:54.775508 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:54.775453    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc76ce0ad24ebf12efa8268f9ba72d035f338f587ca4dbea109328cea036bb71"} err="failed to get container status \"bc76ce0ad24ebf12efa8268f9ba72d035f338f587ca4dbea109328cea036bb71\": rpc error: code = NotFound desc = could not find container \"bc76ce0ad24ebf12efa8268f9ba72d035f338f587ca4dbea109328cea036bb71\": container with ID starting with bc76ce0ad24ebf12efa8268f9ba72d035f338f587ca4dbea109328cea036bb71 not found: ID does not exist"
Jul 23 23:56:54.775508 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:54.775478    2578 scope.go:117] "RemoveContainer" containerID="92108de625269ac67a2674374b54b817f3c8e6821bac437554ac7dc168ad52a7"
Jul 23 23:56:54.775742 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:56:54.775725    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92108de625269ac67a2674374b54b817f3c8e6821bac437554ac7dc168ad52a7\": container with ID starting with 92108de625269ac67a2674374b54b817f3c8e6821bac437554ac7dc168ad52a7 not found: ID does not exist" containerID="92108de625269ac67a2674374b54b817f3c8e6821bac437554ac7dc168ad52a7"
Jul 23 23:56:54.775781 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:54.775749    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92108de625269ac67a2674374b54b817f3c8e6821bac437554ac7dc168ad52a7"} err="failed to get container status \"92108de625269ac67a2674374b54b817f3c8e6821bac437554ac7dc168ad52a7\": rpc error: code = NotFound desc = could not find container \"92108de625269ac67a2674374b54b817f3c8e6821bac437554ac7dc168ad52a7\": container with ID starting with 92108de625269ac67a2674374b54b817f3c8e6821bac437554ac7dc168ad52a7 not found: ID does not exist"
Jul 23 23:56:54.779893 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:54.779870    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8"]
Jul 23 23:56:54.783124 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:54.783101    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-867102-openshift-default-7bfb45d576-5f8k8"]
Jul 23 23:56:55.236636 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:55.236603    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13b2835b-0e85-44a3-a4b1-9172fff9f137" path="/var/lib/kubelet/pods/13b2835b-0e85-44a3-a4b1-9172fff9f137/volumes"
Jul 23 23:56:59.558228 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:59.558191    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-fc333b-589fb55cb5-qjl47"]
Jul 23 23:56:59.558837 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:59.558454    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fc333b-589fb55cb5-qjl47" podUID="0ea0f5cd-26af-474a-b36a-73b1ddb2c501" containerName="maas-api" containerID="cri-o://5dd47cee09b3d311732321ba939a46d4c2c5beb3f88403b8c0bb702a2d5eb599" gracePeriod=30
Jul 23 23:56:59.785287 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:59.785258    2578 generic.go:356] "Generic (PLEG): container finished" podID="0ea0f5cd-26af-474a-b36a-73b1ddb2c501" containerID="5dd47cee09b3d311732321ba939a46d4c2c5beb3f88403b8c0bb702a2d5eb599" exitCode=0
Jul 23 23:56:59.785439 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:59.785334    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fc333b-589fb55cb5-qjl47" event={"ID":"0ea0f5cd-26af-474a-b36a-73b1ddb2c501","Type":"ContainerDied","Data":"5dd47cee09b3d311732321ba939a46d4c2c5beb3f88403b8c0bb702a2d5eb599"}
Jul 23 23:56:59.816460 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:59.816401    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fc333b-589fb55cb5-qjl47"
Jul 23 23:56:59.909214 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:59.909169    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ea0f5cd-26af-474a-b36a-73b1ddb2c501-openshift-service-ca\") pod \"0ea0f5cd-26af-474a-b36a-73b1ddb2c501\" (UID: \"0ea0f5cd-26af-474a-b36a-73b1ddb2c501\") "
Jul 23 23:56:59.909413 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:59.909291    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ea0f5cd-26af-474a-b36a-73b1ddb2c501-maas-api-tls\") pod \"0ea0f5cd-26af-474a-b36a-73b1ddb2c501\" (UID: \"0ea0f5cd-26af-474a-b36a-73b1ddb2c501\") "
Jul 23 23:56:59.909413 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:59.909312    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zpd86\" (UniqueName: \"kubernetes.io/projected/0ea0f5cd-26af-474a-b36a-73b1ddb2c501-kube-api-access-zpd86\") pod \"0ea0f5cd-26af-474a-b36a-73b1ddb2c501\" (UID: \"0ea0f5cd-26af-474a-b36a-73b1ddb2c501\") "
Jul 23 23:56:59.909603 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:59.909579    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ea0f5cd-26af-474a-b36a-73b1ddb2c501-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0ea0f5cd-26af-474a-b36a-73b1ddb2c501" (UID: "0ea0f5cd-26af-474a-b36a-73b1ddb2c501"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:56:59.911544 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:59.911516    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ea0f5cd-26af-474a-b36a-73b1ddb2c501-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0ea0f5cd-26af-474a-b36a-73b1ddb2c501" (UID: "0ea0f5cd-26af-474a-b36a-73b1ddb2c501"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 23:56:59.911676 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:56:59.911632    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ea0f5cd-26af-474a-b36a-73b1ddb2c501-kube-api-access-zpd86" (OuterVolumeSpecName: "kube-api-access-zpd86") pod "0ea0f5cd-26af-474a-b36a-73b1ddb2c501" (UID: "0ea0f5cd-26af-474a-b36a-73b1ddb2c501"). InnerVolumeSpecName "kube-api-access-zpd86". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:57:00.010912 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:00.010882    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ea0f5cd-26af-474a-b36a-73b1ddb2c501-maas-api-tls\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:57:00.010912 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:00.010907    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zpd86\" (UniqueName: \"kubernetes.io/projected/0ea0f5cd-26af-474a-b36a-73b1ddb2c501-kube-api-access-zpd86\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:57:00.010912 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:00.010918    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ea0f5cd-26af-474a-b36a-73b1ddb2c501-openshift-service-ca\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:57:00.791717 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:00.791678    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fc333b-589fb55cb5-qjl47" event={"ID":"0ea0f5cd-26af-474a-b36a-73b1ddb2c501","Type":"ContainerDied","Data":"d63893b82f6b505e78d1f01b7000c85ec40e0169a9217d3973d6238f43e53a04"}
Jul 23 23:57:00.791717 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:00.791708    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fc333b-589fb55cb5-qjl47"
Jul 23 23:57:00.792283 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:00.791766    2578 scope.go:117] "RemoveContainer" containerID="5dd47cee09b3d311732321ba939a46d4c2c5beb3f88403b8c0bb702a2d5eb599"
Jul 23 23:57:00.815264 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:00.815236    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-fc333b-589fb55cb5-qjl47"]
Jul 23 23:57:00.822728 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:00.822696    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-fc333b-589fb55cb5-qjl47"]
Jul 23 23:57:01.236664 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:01.236618    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ea0f5cd-26af-474a-b36a-73b1ddb2c501" path="/var/lib/kubelet/pods/0ea0f5cd-26af-474a-b36a-73b1ddb2c501/volumes"
Jul 23 23:57:10.525214 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:10.525174    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh"]
Jul 23 23:57:10.525681 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:10.525435    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh" podUID="e8fbe86f-2de9-44ef-861e-70362bd1476e" containerName="istio-proxy" containerID="cri-o://45fc908f232f42f8e38c4b48af73829544b69b600beb0c5e3f8e88dba6b0ebd0" gracePeriod=30
Jul 23 23:57:12.146936 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:12.146892    2578 patch_prober.go:28] interesting pod/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 23 23:57:12.147308 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:12.146951    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh" podUID="e8fbe86f-2de9-44ef-861e-70362bd1476e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 23 23:57:15.780049 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:15.780025    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh"
Jul 23 23:57:15.864970 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:15.864880    2578 generic.go:356] "Generic (PLEG): container finished" podID="e8fbe86f-2de9-44ef-861e-70362bd1476e" containerID="45fc908f232f42f8e38c4b48af73829544b69b600beb0c5e3f8e88dba6b0ebd0" exitCode=0
Jul 23 23:57:15.864970 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:15.864931    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh"
Jul 23 23:57:15.865144 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:15.864964    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh" event={"ID":"e8fbe86f-2de9-44ef-861e-70362bd1476e","Type":"ContainerDied","Data":"45fc908f232f42f8e38c4b48af73829544b69b600beb0c5e3f8e88dba6b0ebd0"}
Jul 23 23:57:15.865144 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:15.865021    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh" event={"ID":"e8fbe86f-2de9-44ef-861e-70362bd1476e","Type":"ContainerDied","Data":"148030f15a85bc921587c7f2ab15ce86e87e0b35035b02a5e51e6948b1ef7553"}
Jul 23 23:57:15.865144 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:15.865046    2578 scope.go:117] "RemoveContainer" containerID="45fc908f232f42f8e38c4b48af73829544b69b600beb0c5e3f8e88dba6b0ebd0"
Jul 23 23:57:15.867216 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:15.867200    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8fbe86f-2de9-44ef-861e-70362bd1476e-istio-token\") pod \"e8fbe86f-2de9-44ef-861e-70362bd1476e\" (UID: \"e8fbe86f-2de9-44ef-861e-70362bd1476e\") "
Jul 23 23:57:15.867317 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:15.867226    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-czxkc\" (UniqueName: \"kubernetes.io/projected/e8fbe86f-2de9-44ef-861e-70362bd1476e-kube-api-access-czxkc\") pod \"e8fbe86f-2de9-44ef-861e-70362bd1476e\" (UID: \"e8fbe86f-2de9-44ef-861e-70362bd1476e\") "
Jul 23 23:57:15.867317 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:15.867266    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8fbe86f-2de9-44ef-861e-70362bd1476e-istio-podinfo\") pod \"e8fbe86f-2de9-44ef-861e-70362bd1476e\" (UID: \"e8fbe86f-2de9-44ef-861e-70362bd1476e\") "
Jul 23 23:57:15.867317 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:15.867287    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8fbe86f-2de9-44ef-861e-70362bd1476e-istio-envoy\") pod \"e8fbe86f-2de9-44ef-861e-70362bd1476e\" (UID: \"e8fbe86f-2de9-44ef-861e-70362bd1476e\") "
Jul 23 23:57:15.867485 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:15.867401    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8fbe86f-2de9-44ef-861e-70362bd1476e-workload-socket\") pod \"e8fbe86f-2de9-44ef-861e-70362bd1476e\" (UID: \"e8fbe86f-2de9-44ef-861e-70362bd1476e\") "
Jul 23 23:57:15.867485 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:15.867461    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8fbe86f-2de9-44ef-861e-70362bd1476e-credential-socket\") pod \"e8fbe86f-2de9-44ef-861e-70362bd1476e\" (UID: \"e8fbe86f-2de9-44ef-861e-70362bd1476e\") "
Jul 23 23:57:15.867628 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:15.867566    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8fbe86f-2de9-44ef-861e-70362bd1476e-istiod-ca-cert\") pod \"e8fbe86f-2de9-44ef-861e-70362bd1476e\" (UID: \"e8fbe86f-2de9-44ef-861e-70362bd1476e\") "
Jul 23 23:57:15.867628 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:15.867606    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8fbe86f-2de9-44ef-861e-70362bd1476e-workload-certs\") pod \"e8fbe86f-2de9-44ef-861e-70362bd1476e\" (UID: \"e8fbe86f-2de9-44ef-861e-70362bd1476e\") "
Jul 23 23:57:15.867771 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:15.867634    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8fbe86f-2de9-44ef-861e-70362bd1476e-istio-data\") pod \"e8fbe86f-2de9-44ef-861e-70362bd1476e\" (UID: \"e8fbe86f-2de9-44ef-861e-70362bd1476e\") "
Jul 23 23:57:15.867771 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:15.867691    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8fbe86f-2de9-44ef-861e-70362bd1476e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e8fbe86f-2de9-44ef-861e-70362bd1476e" (UID: "e8fbe86f-2de9-44ef-861e-70362bd1476e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:57:15.868137 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:15.867901    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8fbe86f-2de9-44ef-861e-70362bd1476e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e8fbe86f-2de9-44ef-861e-70362bd1476e" (UID: "e8fbe86f-2de9-44ef-861e-70362bd1476e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:57:15.868137 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:15.867894    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8fbe86f-2de9-44ef-861e-70362bd1476e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e8fbe86f-2de9-44ef-861e-70362bd1476e" (UID: "e8fbe86f-2de9-44ef-861e-70362bd1476e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:57:15.868137 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:15.867974    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8fbe86f-2de9-44ef-861e-70362bd1476e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e8fbe86f-2de9-44ef-861e-70362bd1476e" (UID: "e8fbe86f-2de9-44ef-861e-70362bd1476e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 23:57:15.868137 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:15.867994    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8fbe86f-2de9-44ef-861e-70362bd1476e-workload-socket\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:57:15.868137 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:15.868014    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8fbe86f-2de9-44ef-861e-70362bd1476e-credential-socket\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:57:15.868137 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:15.868028    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8fbe86f-2de9-44ef-861e-70362bd1476e-workload-certs\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:57:15.868673 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:15.868626    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8fbe86f-2de9-44ef-861e-70362bd1476e-istio-data" (OuterVolumeSpecName: "istio-data") pod "e8fbe86f-2de9-44ef-861e-70362bd1476e" (UID: "e8fbe86f-2de9-44ef-861e-70362bd1476e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:57:15.869629 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:15.869608    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8fbe86f-2de9-44ef-861e-70362bd1476e-kube-api-access-czxkc" (OuterVolumeSpecName: "kube-api-access-czxkc") pod "e8fbe86f-2de9-44ef-861e-70362bd1476e" (UID: "e8fbe86f-2de9-44ef-861e-70362bd1476e"). InnerVolumeSpecName "kube-api-access-czxkc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:57:15.870087 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:15.870060    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e8fbe86f-2de9-44ef-861e-70362bd1476e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e8fbe86f-2de9-44ef-861e-70362bd1476e" (UID: "e8fbe86f-2de9-44ef-861e-70362bd1476e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 23:57:15.870183 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:15.870111    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8fbe86f-2de9-44ef-861e-70362bd1476e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e8fbe86f-2de9-44ef-861e-70362bd1476e" (UID: "e8fbe86f-2de9-44ef-861e-70362bd1476e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 23:57:15.870330 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:15.870304    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8fbe86f-2de9-44ef-861e-70362bd1476e-istio-token" (OuterVolumeSpecName: "istio-token") pod "e8fbe86f-2de9-44ef-861e-70362bd1476e" (UID: "e8fbe86f-2de9-44ef-861e-70362bd1476e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 23:57:15.888770 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:15.888727    2578 scope.go:117] "RemoveContainer" containerID="07c977f76601de18ecfe7ef41c4c4a96f4b0556eb049a70efbc9697c18c86f2d"
Jul 23 23:57:15.896854 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:15.896833    2578 scope.go:117] "RemoveContainer" containerID="45fc908f232f42f8e38c4b48af73829544b69b600beb0c5e3f8e88dba6b0ebd0"
Jul 23 23:57:15.897133 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:57:15.897115    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45fc908f232f42f8e38c4b48af73829544b69b600beb0c5e3f8e88dba6b0ebd0\": container with ID starting with 45fc908f232f42f8e38c4b48af73829544b69b600beb0c5e3f8e88dba6b0ebd0 not found: ID does not exist" containerID="45fc908f232f42f8e38c4b48af73829544b69b600beb0c5e3f8e88dba6b0ebd0"
Jul 23 23:57:15.897180 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:15.897142    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45fc908f232f42f8e38c4b48af73829544b69b600beb0c5e3f8e88dba6b0ebd0"} err="failed to get container status \"45fc908f232f42f8e38c4b48af73829544b69b600beb0c5e3f8e88dba6b0ebd0\": rpc error: code = NotFound desc = could not find container \"45fc908f232f42f8e38c4b48af73829544b69b600beb0c5e3f8e88dba6b0ebd0\": container with ID starting with 45fc908f232f42f8e38c4b48af73829544b69b600beb0c5e3f8e88dba6b0ebd0 not found: ID does not exist"
Jul 23 23:57:15.897180 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:15.897160    2578 scope.go:117] "RemoveContainer" containerID="07c977f76601de18ecfe7ef41c4c4a96f4b0556eb049a70efbc9697c18c86f2d"
Jul 23 23:57:15.897401 ip-10-0-129-165 kubenswrapper[2578]: E0723 23:57:15.897383    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07c977f76601de18ecfe7ef41c4c4a96f4b0556eb049a70efbc9697c18c86f2d\": container with ID starting with 07c977f76601de18ecfe7ef41c4c4a96f4b0556eb049a70efbc9697c18c86f2d not found: ID does not exist" containerID="07c977f76601de18ecfe7ef41c4c4a96f4b0556eb049a70efbc9697c18c86f2d"
Jul 23 23:57:15.897444 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:15.897407    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07c977f76601de18ecfe7ef41c4c4a96f4b0556eb049a70efbc9697c18c86f2d"} err="failed to get container status \"07c977f76601de18ecfe7ef41c4c4a96f4b0556eb049a70efbc9697c18c86f2d\": rpc error: code = NotFound desc = could not find container \"07c977f76601de18ecfe7ef41c4c4a96f4b0556eb049a70efbc9697c18c86f2d\": container with ID starting with 07c977f76601de18ecfe7ef41c4c4a96f4b0556eb049a70efbc9697c18c86f2d not found: ID does not exist"
Jul 23 23:57:15.969272 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:15.969233    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8fbe86f-2de9-44ef-861e-70362bd1476e-istiod-ca-cert\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:57:15.969272 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:15.969265    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8fbe86f-2de9-44ef-861e-70362bd1476e-istio-data\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:57:15.969272 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:15.969273    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8fbe86f-2de9-44ef-861e-70362bd1476e-istio-token\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:57:15.969272 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:15.969282    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-czxkc\" (UniqueName: \"kubernetes.io/projected/e8fbe86f-2de9-44ef-861e-70362bd1476e-kube-api-access-czxkc\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:57:15.969587 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:15.969293    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8fbe86f-2de9-44ef-861e-70362bd1476e-istio-podinfo\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:57:15.969587 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:15.969302    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8fbe86f-2de9-44ef-861e-70362bd1476e-istio-envoy\") on node \"ip-10-0-129-165.ec2.internal\" DevicePath \"\""
Jul 23 23:57:16.186885 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:16.186844    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh"]
Jul 23 23:57:16.190670 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:16.190623    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-fc333b-openshift-default-5cfc7d6dd-lw8wh"]
Jul 23 23:57:17.236734 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:17.236697    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8fbe86f-2de9-44ef-861e-70362bd1476e" path="/var/lib/kubelet/pods/e8fbe86f-2de9-44ef-861e-70362bd1476e/volumes"
Jul 23 23:57:17.568671 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:17.568566    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5fd4b8775f-mqt45_67c70406-7b85-41ee-8e6d-12b6714bd46d/copy-manifests/0.log"
Jul 23 23:57:17.575185 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:17.575155    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5fd4b8775f-mqt45_67c70406-7b85-41ee-8e6d-12b6714bd46d/manager/0.log"
Jul 23 23:57:18.058754 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:18.058714    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-7d567cf478-jwp69_50a3bff4-f9bb-49eb-bc39-fa9206bb6472/maas-api/0.log"
Jul 23 23:57:18.170189 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:18.170157    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29747490-97mvp_fc5db127-44bc-4f40-86de-b9c54d323591/cleanup/0.log"
Jul 23 23:57:18.283677 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:18.283621    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29747505-bgd7q_1725205b-76b9-4902-989f-50ed2e38a3f3/cleanup/0.log"
Jul 23 23:57:20.958293 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:20.958255    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp_a4107bfc-1c4c-4d54-a70c-b0e65df49887/istio-proxy/0.log"
Jul 23 23:57:21.088938 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:21.088904    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-bv462_ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022/discovery/0.log"
Jul 23 23:57:21.457898 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:21.457854    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-df6776dd6-7jzmm_e01c2593-7e05-4f54-83db-4ac9af47dda1/istio-proxy/1.log"
Jul 23 23:57:21.597923 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:21.597888    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-mhd8c_23725880-de32-4467-b1f6-8e797fabb616/payload-pre-processing/0.log"
Jul 23 23:57:22.302420 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:22.302391    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-8697q_48538ffa-b3c4-4866-b1cc-5d92defabff2/main/0.log"
Jul 23 23:57:22.428272 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:22.428239    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-g6cjv_d3430807-c44a-4b2a-9ce5-3f1c38a246dc/main/0.log"
Jul 23 23:57:31.499416 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:31.499382    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-z8d42_b4d53a59-83bb-4725-841c-9f5b55f85ca4/global-pull-secret-syncer/0.log"
Jul 23 23:57:31.580899 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:31.580869    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-9jn86_960f9613-6e9d-4172-a662-b20bc07d8705/konnectivity-agent/0.log"
Jul 23 23:57:31.645521 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:31.645482    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-165.ec2.internal_f2a7eb7430412c2da89feeebacf09842/haproxy/0.log"
Jul 23 23:57:39.501603 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:39.501564    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-v29tt_9ee7a305-ec45-4c90-9033-05158464f844/kube-state-metrics/0.log"
Jul 23 23:57:39.523390 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:39.523366    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-v29tt_9ee7a305-ec45-4c90-9033-05158464f844/kube-rbac-proxy-main/0.log"
Jul 23 23:57:39.540912 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:39.540883    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-v29tt_9ee7a305-ec45-4c90-9033-05158464f844/kube-rbac-proxy-self/0.log"
Jul 23 23:57:39.564513 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:39.564488    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6d677d6cd8-w9t7b_a805bdfd-024e-4561-8845-14476580ce6e/metrics-server/0.log"
Jul 23 23:57:39.786070 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:39.785979    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wcbw2_426f9a19-5c8c-4c1c-aaba-fc30325db699/node-exporter/0.log"
Jul 23 23:57:39.819688 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:39.819643    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wcbw2_426f9a19-5c8c-4c1c-aaba-fc30325db699/kube-rbac-proxy/0.log"
Jul 23 23:57:39.837979 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:39.837956    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wcbw2_426f9a19-5c8c-4c1c-aaba-fc30325db699/init-textfile/0.log"
Jul 23 23:57:39.865030 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:39.865006    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-6c8qv_292b6375-d147-4097-8eef-61692f076e3a/kube-rbac-proxy-main/0.log"
Jul 23 23:57:39.882543 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:39.882513    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-6c8qv_292b6375-d147-4097-8eef-61692f076e3a/kube-rbac-proxy-self/0.log"
Jul 23 23:57:39.901367 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:39.901339    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-6c8qv_292b6375-d147-4097-8eef-61692f076e3a/openshift-state-metrics/0.log"
Jul 23 23:57:39.933968 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:39.933942    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5852f69c-c825-45e2-b05e-29c7ac1d7f82/prometheus/0.log"
Jul 23 23:57:39.951805 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:39.951773    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5852f69c-c825-45e2-b05e-29c7ac1d7f82/config-reloader/0.log"
Jul 23 23:57:39.969403 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:39.969381    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5852f69c-c825-45e2-b05e-29c7ac1d7f82/thanos-sidecar/0.log"
Jul 23 23:57:39.988057 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:39.988025    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5852f69c-c825-45e2-b05e-29c7ac1d7f82/kube-rbac-proxy-web/0.log"
Jul 23 23:57:40.007099 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:40.007078    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5852f69c-c825-45e2-b05e-29c7ac1d7f82/kube-rbac-proxy/0.log"
Jul 23 23:57:40.024043 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:40.024022    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5852f69c-c825-45e2-b05e-29c7ac1d7f82/kube-rbac-proxy-thanos/0.log"
Jul 23 23:57:40.042597 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:40.042530    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5852f69c-c825-45e2-b05e-29c7ac1d7f82/init-config-reloader/0.log"
Jul 23 23:57:40.069873 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:40.069847    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-dp2w9_0fcfa80a-1df2-4525-8b6e-2ca734dcead8/prometheus-operator/0.log"
Jul 23 23:57:40.084760 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:40.084738    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-dp2w9_0fcfa80a-1df2-4525-8b6e-2ca734dcead8/kube-rbac-proxy/0.log"
Jul 23 23:57:40.104248 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:40.104227    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-plx4r_3d2457d7-2cbe-4d01-ae77-d89c4a280df2/prometheus-operator-admission-webhook/0.log"
Jul 23 23:57:40.129307 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:40.129278    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-69ccdd5d84-8kg6n_795324dd-aad0-42c7-a072-45748bc9f06f/telemeter-client/0.log"
Jul 23 23:57:40.147349 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:40.147320    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-69ccdd5d84-8kg6n_795324dd-aad0-42c7-a072-45748bc9f06f/reload/0.log"
Jul 23 23:57:40.172945 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:40.172926    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-69ccdd5d84-8kg6n_795324dd-aad0-42c7-a072-45748bc9f06f/kube-rbac-proxy/0.log"
Jul 23 23:57:41.782849 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:41.782811    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gm46q/perf-node-gather-daemonset-85w8l"]
Jul 23 23:57:41.783744 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:41.783637    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8fbe86f-2de9-44ef-861e-70362bd1476e" containerName="istio-proxy"
Jul 23 23:57:41.783744 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:41.783680    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8fbe86f-2de9-44ef-861e-70362bd1476e" containerName="istio-proxy"
Jul 23 23:57:41.783744 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:41.783701    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13b2835b-0e85-44a3-a4b1-9172fff9f137" containerName="istio-proxy"
Jul 23 23:57:41.783744 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:41.783710    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="13b2835b-0e85-44a3-a4b1-9172fff9f137" containerName="istio-proxy"
Jul 23 23:57:41.783744 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:41.783724    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13b2835b-0e85-44a3-a4b1-9172fff9f137" containerName="istio-proxy"
Jul 23 23:57:41.783744 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:41.783732    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="13b2835b-0e85-44a3-a4b1-9172fff9f137" containerName="istio-proxy"
Jul 23 23:57:41.783744 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:41.783742    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ea0f5cd-26af-474a-b36a-73b1ddb2c501" containerName="maas-api"
Jul 23 23:57:41.784202 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:41.783752    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ea0f5cd-26af-474a-b36a-73b1ddb2c501" containerName="maas-api"
Jul 23 23:57:41.784202 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:41.783763    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8fbe86f-2de9-44ef-861e-70362bd1476e" containerName="istio-proxy"
Jul 23 23:57:41.784202 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:41.783771    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8fbe86f-2de9-44ef-861e-70362bd1476e" containerName="istio-proxy"
Jul 23 23:57:41.784202 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:41.783785    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0823b6c-309c-45de-95d6-6548497df078" containerName="maas-api"
Jul 23 23:57:41.784202 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:41.783793    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0823b6c-309c-45de-95d6-6548497df078" containerName="maas-api"
Jul 23 23:57:41.784202 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:41.783883    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="13b2835b-0e85-44a3-a4b1-9172fff9f137" containerName="istio-proxy"
Jul 23 23:57:41.784202 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:41.783893    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8fbe86f-2de9-44ef-861e-70362bd1476e" containerName="istio-proxy"
Jul 23 23:57:41.784202 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:41.783905    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0823b6c-309c-45de-95d6-6548497df078" containerName="maas-api"
Jul 23 23:57:41.784202 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:41.783913    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="13b2835b-0e85-44a3-a4b1-9172fff9f137" containerName="istio-proxy"
Jul 23 23:57:41.784202 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:41.783921    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ea0f5cd-26af-474a-b36a-73b1ddb2c501" containerName="maas-api"
Jul 23 23:57:41.787142 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:41.787121    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gm46q/perf-node-gather-daemonset-85w8l"
Jul 23 23:57:41.790018 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:41.789692    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gm46q\"/\"openshift-service-ca.crt\""
Jul 23 23:57:41.790018 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:41.789779    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gm46q\"/\"kube-root-ca.crt\""
Jul 23 23:57:41.790938 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:41.790922    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gm46q\"/\"default-dockercfg-m5j7c\""
Jul 23 23:57:41.795898 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:41.795875    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gm46q/perf-node-gather-daemonset-85w8l"]
Jul 23 23:57:41.802279 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:41.802247    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c82aae22-9e47-4118-9f76-685a586f63d5-lib-modules\") pod \"perf-node-gather-daemonset-85w8l\" (UID: \"c82aae22-9e47-4118-9f76-685a586f63d5\") " pod="openshift-must-gather-gm46q/perf-node-gather-daemonset-85w8l"
Jul 23 23:57:41.802395 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:41.802318    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldhpq\" (UniqueName: \"kubernetes.io/projected/c82aae22-9e47-4118-9f76-685a586f63d5-kube-api-access-ldhpq\") pod \"perf-node-gather-daemonset-85w8l\" (UID: \"c82aae22-9e47-4118-9f76-685a586f63d5\") " pod="openshift-must-gather-gm46q/perf-node-gather-daemonset-85w8l"
Jul 23 23:57:41.802492 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:41.802470    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c82aae22-9e47-4118-9f76-685a586f63d5-podres\") pod \"perf-node-gather-daemonset-85w8l\" (UID: \"c82aae22-9e47-4118-9f76-685a586f63d5\") " pod="openshift-must-gather-gm46q/perf-node-gather-daemonset-85w8l"
Jul 23 23:57:41.802551 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:41.802535    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c82aae22-9e47-4118-9f76-685a586f63d5-sys\") pod \"perf-node-gather-daemonset-85w8l\" (UID: \"c82aae22-9e47-4118-9f76-685a586f63d5\") " pod="openshift-must-gather-gm46q/perf-node-gather-daemonset-85w8l"
Jul 23 23:57:41.802623 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:41.802593    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c82aae22-9e47-4118-9f76-685a586f63d5-proc\") pod \"perf-node-gather-daemonset-85w8l\" (UID: \"c82aae22-9e47-4118-9f76-685a586f63d5\") " pod="openshift-must-gather-gm46q/perf-node-gather-daemonset-85w8l"
Jul 23 23:57:41.903302 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:41.903263    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c82aae22-9e47-4118-9f76-685a586f63d5-podres\") pod \"perf-node-gather-daemonset-85w8l\" (UID: \"c82aae22-9e47-4118-9f76-685a586f63d5\") " pod="openshift-must-gather-gm46q/perf-node-gather-daemonset-85w8l"
Jul 23 23:57:41.903302 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:41.903307    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c82aae22-9e47-4118-9f76-685a586f63d5-sys\") pod \"perf-node-gather-daemonset-85w8l\" (UID: \"c82aae22-9e47-4118-9f76-685a586f63d5\") " pod="openshift-must-gather-gm46q/perf-node-gather-daemonset-85w8l"
Jul 23 23:57:41.903524 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:41.903329    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c82aae22-9e47-4118-9f76-685a586f63d5-proc\") pod \"perf-node-gather-daemonset-85w8l\" (UID: \"c82aae22-9e47-4118-9f76-685a586f63d5\") " pod="openshift-must-gather-gm46q/perf-node-gather-daemonset-85w8l"
Jul 23 23:57:41.903524 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:41.903366    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c82aae22-9e47-4118-9f76-685a586f63d5-lib-modules\") pod \"perf-node-gather-daemonset-85w8l\" (UID: \"c82aae22-9e47-4118-9f76-685a586f63d5\") " pod="openshift-must-gather-gm46q/perf-node-gather-daemonset-85w8l"
Jul 23 23:57:41.903524 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:41.903402    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldhpq\" (UniqueName: \"kubernetes.io/projected/c82aae22-9e47-4118-9f76-685a586f63d5-kube-api-access-ldhpq\") pod \"perf-node-gather-daemonset-85w8l\" (UID: \"c82aae22-9e47-4118-9f76-685a586f63d5\") " pod="openshift-must-gather-gm46q/perf-node-gather-daemonset-85w8l"
Jul 23 23:57:41.903524 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:41.903408    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c82aae22-9e47-4118-9f76-685a586f63d5-sys\") pod \"perf-node-gather-daemonset-85w8l\" (UID: \"c82aae22-9e47-4118-9f76-685a586f63d5\") " pod="openshift-must-gather-gm46q/perf-node-gather-daemonset-85w8l"
Jul 23 23:57:41.903524 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:41.903432    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c82aae22-9e47-4118-9f76-685a586f63d5-podres\") pod \"perf-node-gather-daemonset-85w8l\" (UID: \"c82aae22-9e47-4118-9f76-685a586f63d5\") " pod="openshift-must-gather-gm46q/perf-node-gather-daemonset-85w8l"
Jul 23 23:57:41.903524 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:41.903476    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c82aae22-9e47-4118-9f76-685a586f63d5-proc\") pod \"perf-node-gather-daemonset-85w8l\" (UID: \"c82aae22-9e47-4118-9f76-685a586f63d5\") " pod="openshift-must-gather-gm46q/perf-node-gather-daemonset-85w8l"
Jul 23 23:57:41.903783 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:41.903540    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c82aae22-9e47-4118-9f76-685a586f63d5-lib-modules\") pod \"perf-node-gather-daemonset-85w8l\" (UID: \"c82aae22-9e47-4118-9f76-685a586f63d5\") " pod="openshift-must-gather-gm46q/perf-node-gather-daemonset-85w8l"
Jul 23 23:57:41.911363 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:41.911341    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldhpq\" (UniqueName: \"kubernetes.io/projected/c82aae22-9e47-4118-9f76-685a586f63d5-kube-api-access-ldhpq\") pod \"perf-node-gather-daemonset-85w8l\" (UID: \"c82aae22-9e47-4118-9f76-685a586f63d5\") " pod="openshift-must-gather-gm46q/perf-node-gather-daemonset-85w8l"
Jul 23 23:57:42.099908 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:42.099819    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gm46q/perf-node-gather-daemonset-85w8l"
Jul 23 23:57:42.233605 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:42.233421    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gm46q/perf-node-gather-daemonset-85w8l"]
Jul 23 23:57:42.236836 ip-10-0-129-165 kubenswrapper[2578]: W0723 23:57:42.236803    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc82aae22_9e47_4118_9f76_685a586f63d5.slice/crio-7e5ac85c0909b054739d0c90fafdd6b15dfaa6cc8ff9b82938df02625a30a165 WatchSource:0}: Error finding container 7e5ac85c0909b054739d0c90fafdd6b15dfaa6cc8ff9b82938df02625a30a165: Status 404 returned error can't find the container with id 7e5ac85c0909b054739d0c90fafdd6b15dfaa6cc8ff9b82938df02625a30a165
Jul 23 23:57:42.991529 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:42.991492    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gm46q/perf-node-gather-daemonset-85w8l" event={"ID":"c82aae22-9e47-4118-9f76-685a586f63d5","Type":"ContainerStarted","Data":"70c44dbf7f726176a1ab455db27ed32fa4684c482b6ef4ef07873d1de6aa3960"}
Jul 23 23:57:42.991937 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:42.991549    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gm46q/perf-node-gather-daemonset-85w8l" event={"ID":"c82aae22-9e47-4118-9f76-685a586f63d5","Type":"ContainerStarted","Data":"7e5ac85c0909b054739d0c90fafdd6b15dfaa6cc8ff9b82938df02625a30a165"}
Jul 23 23:57:42.991937 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:42.991592    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-gm46q/perf-node-gather-daemonset-85w8l"
Jul 23 23:57:43.007985 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:43.007937    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gm46q/perf-node-gather-daemonset-85w8l" podStartSLOduration=2.007922662 podStartE2EDuration="2.007922662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 23:57:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 23:57:43.006282229 +0000 UTC m=+2652.375807899" watchObservedRunningTime="2026-07-23 23:57:43.007922662 +0000 UTC m=+2652.377448320"
Jul 23 23:57:44.330829 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:44.330794    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9bpqj_0171d6e4-8fda-499c-83a3-b0ec8ec597d1/dns/0.log"
Jul 23 23:57:44.347058 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:44.347034    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9bpqj_0171d6e4-8fda-499c-83a3-b0ec8ec597d1/kube-rbac-proxy/0.log"
Jul 23 23:57:44.394470 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:44.394440    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sn5k2_6c30a49c-1943-4650-83f0-adaa3045404e/dns-node-resolver/0.log"
Jul 23 23:57:44.940404 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:44.940373    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ljq8l_d2943613-62a6-476e-bec2-12fb3c167e01/node-ca/0.log"
Jul 23 23:57:45.969727 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:45.969694    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp_a4107bfc-1c4c-4d54-a70c-b0e65df49887/istio-proxy/0.log"
Jul 23 23:57:46.187579 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:46.187541    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-bv462_ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022/discovery/0.log"
Jul 23 23:57:46.255155 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:46.255065    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-df6776dd6-7jzmm_e01c2593-7e05-4f54-83db-4ac9af47dda1/istio-proxy/0.log"
Jul 23 23:57:46.306310 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:46.306270    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-df6776dd6-7jzmm_e01c2593-7e05-4f54-83db-4ac9af47dda1/istio-proxy/1.log"
Jul 23 23:57:46.344631 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:46.344594    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-mhd8c_23725880-de32-4467-b1f6-8e797fabb616/payload-pre-processing/0.log"
Jul 23 23:57:47.079328 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:47.079295    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hsq95_db753c50-9066-482f-88d2-94dc1b55de52/serve-healthcheck-canary/0.log"
Jul 23 23:57:47.615351 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:47.615312    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6pgdz_b7125927-8aa3-4326-b1f2-8950ca674b39/kube-rbac-proxy/0.log"
Jul 23 23:57:47.632612 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:47.632574    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6pgdz_b7125927-8aa3-4326-b1f2-8950ca674b39/exporter/0.log"
Jul 23 23:57:47.651798 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:47.651775    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6pgdz_b7125927-8aa3-4326-b1f2-8950ca674b39/extractor/0.log"
Jul 23 23:57:49.006224 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:49.006197    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-gm46q/perf-node-gather-daemonset-85w8l"
Jul 23 23:57:50.513227 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:50.513165    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5fd4b8775f-mqt45_67c70406-7b85-41ee-8e6d-12b6714bd46d/manager/0.log"
Jul 23 23:57:50.533643 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:50.533616    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5fd4b8775f-mqt45_67c70406-7b85-41ee-8e6d-12b6714bd46d/copy-manifests/0.log"
Jul 23 23:57:52.023929 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:52.023892    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-68dbcfb44d-wqzsq_fa8bf31f-5813-4360-9d34-8101f443652b/manager/0.log"
Jul 23 23:57:52.037328 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:52.037291    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp_a4107bfc-1c4c-4d54-a70c-b0e65df49887/istio-proxy/0.log"
Jul 23 23:57:52.045046 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:52.045018    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-lr24w_22e74d99-a1bc-4fb2-b0bf-d6371dd1b138/openshift-lws-operator/0.log"
Jul 23 23:57:52.245731 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:52.245698    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-bv462_ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022/discovery/0.log"
Jul 23 23:57:52.297551 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:52.297457    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-df6776dd6-7jzmm_e01c2593-7e05-4f54-83db-4ac9af47dda1/istio-proxy/0.log"
Jul 23 23:57:52.350250 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:52.350218    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-df6776dd6-7jzmm_e01c2593-7e05-4f54-83db-4ac9af47dda1/istio-proxy/1.log"
Jul 23 23:57:52.382272 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:52.382246    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-mhd8c_23725880-de32-4467-b1f6-8e797fabb616/payload-pre-processing/0.log"
Jul 23 23:57:53.553823 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:53.553790    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7b75d5c99cm2zrp_a4107bfc-1c4c-4d54-a70c-b0e65df49887/istio-proxy/0.log"
Jul 23 23:57:53.780003 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:53.779968    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-bv462_ffb8bce6-e2dd-4d5e-8650-c0fa28f0b022/discovery/0.log"
Jul 23 23:57:53.826248 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:53.826154    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-df6776dd6-7jzmm_e01c2593-7e05-4f54-83db-4ac9af47dda1/istio-proxy/0.log"
Jul 23 23:57:53.879024 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:53.878988    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-df6776dd6-7jzmm_e01c2593-7e05-4f54-83db-4ac9af47dda1/istio-proxy/1.log"
Jul 23 23:57:53.911263 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:53.911230    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-mhd8c_23725880-de32-4467-b1f6-8e797fabb616/payload-pre-processing/0.log"
Jul 23 23:57:58.087983 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:58.087901    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rw58q_18ba94f3-1bd7-41f4-a603-0a2654475a66/kube-multus-additional-cni-plugins/0.log"
Jul 23 23:57:58.105110 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:58.105083    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rw58q_18ba94f3-1bd7-41f4-a603-0a2654475a66/egress-router-binary-copy/0.log"
Jul 23 23:57:58.122120 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:58.122092    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rw58q_18ba94f3-1bd7-41f4-a603-0a2654475a66/cni-plugins/0.log"
Jul 23 23:57:58.140284 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:58.140254    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rw58q_18ba94f3-1bd7-41f4-a603-0a2654475a66/bond-cni-plugin/0.log"
Jul 23 23:57:58.158302 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:58.158274    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rw58q_18ba94f3-1bd7-41f4-a603-0a2654475a66/routeoverride-cni/0.log"
Jul 23 23:57:58.175808 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:58.175784    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rw58q_18ba94f3-1bd7-41f4-a603-0a2654475a66/whereabouts-cni-bincopy/0.log"
Jul 23 23:57:58.192371 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:58.192338    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rw58q_18ba94f3-1bd7-41f4-a603-0a2654475a66/whereabouts-cni/0.log"
Jul 23 23:57:58.551039 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:58.551008    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-f66wp_59b943fa-86eb-453f-8e2f-63c64ce435ce/kube-multus/0.log"
Jul 23 23:57:58.572058 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:58.572023    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-bbv68_8dd112f5-f0cc-4e72-917c-33e4abca79cb/network-metrics-daemon/0.log"
Jul 23 23:57:58.588497 ip-10-0-129-165 kubenswrapper[2578]: I0723 23:57:58.588475    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-bbv68_8dd112f5-f0cc-4e72-917c-33e4abca79cb/kube-rbac-proxy/0.log"