Sep 17 14:02:52.674691 ip-10-0-137-53 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 17 14:02:52.674703 ip-10-0-137-53 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 17 14:02:52.674710 ip-10-0-137-53 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 17 14:02:52.674857 ip-10-0-137-53 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 17 14:03:02.892535 ip-10-0-137-53 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 17 14:03:02.892556 ip-10-0-137-53 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 9d0cfabb3ece4594af2290de5390f78b --
Sep 17 14:04:58.889938 ip-10-0-137-53 systemd[1]: Starting Kubernetes Kubelet...
Sep 17 14:04:59.256271 ip-10-0-137-53 kubenswrapper[2582]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 14:04:59.256271 ip-10-0-137-53 kubenswrapper[2582]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 17 14:04:59.256271 ip-10-0-137-53 kubenswrapper[2582]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 14:04:59.256271 ip-10-0-137-53 kubenswrapper[2582]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 17 14:04:59.256271 ip-10-0-137-53 kubenswrapper[2582]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 14:04:59.258586 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.258493    2582 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 17 14:04:59.263892 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.263877    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 14:04:59.263892 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.263892    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 14:04:59.263952 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.263896    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 14:04:59.263952 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.263901    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 14:04:59.263952 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.263905    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 14:04:59.263952 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.263908    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 14:04:59.263952 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.263911    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 14:04:59.263952 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.263914    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 14:04:59.263952 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.263925    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 14:04:59.263952 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.263928    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 14:04:59.263952 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.263931    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 14:04:59.263952 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.263933    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 14:04:59.263952 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.263936    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 14:04:59.263952 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.263939    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 14:04:59.263952 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.263941    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 14:04:59.263952 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.263945    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 14:04:59.263952 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.263947    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 14:04:59.263952 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.263950    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 14:04:59.263952 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.263953    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 14:04:59.263952 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.263955    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 14:04:59.263952 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.263958    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 14:04:59.264423 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.263961    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 14:04:59.264423 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.263964    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 14:04:59.264423 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.263967    2582 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 14:04:59.264423 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.263970    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 14:04:59.264423 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.263972    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 14:04:59.264423 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.263975    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 14:04:59.264423 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.263978    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 14:04:59.264423 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.263980    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 14:04:59.264423 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.263983    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 14:04:59.264423 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.263986    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 14:04:59.264423 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.263989    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 14:04:59.264423 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.263991    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 14:04:59.264423 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.263996    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 14:04:59.264423 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.263999    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 14:04:59.264423 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264003    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 14:04:59.264423 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264006    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 14:04:59.264423 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264008    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 14:04:59.264423 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264011    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 14:04:59.264423 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264014    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 14:04:59.264423 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264017    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 14:04:59.264968 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264020    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 14:04:59.264968 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264022    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 14:04:59.264968 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264024    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 14:04:59.264968 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264027    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 14:04:59.264968 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264029    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 14:04:59.264968 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264032    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 14:04:59.264968 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264034    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 14:04:59.264968 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264037    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 14:04:59.264968 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264039    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 14:04:59.264968 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264042    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 14:04:59.264968 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264044    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 14:04:59.264968 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264047    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 14:04:59.264968 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264049    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 14:04:59.264968 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264054    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 14:04:59.264968 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264056    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 14:04:59.264968 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264059    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 14:04:59.264968 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264062    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 14:04:59.264968 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264064    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 14:04:59.264968 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264067    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 14:04:59.264968 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264069    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 14:04:59.265455 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264072    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 14:04:59.265455 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264074    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 14:04:59.265455 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264077    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 14:04:59.265455 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264080    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 14:04:59.265455 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264082    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 14:04:59.265455 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264085    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 14:04:59.265455 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264089    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 14:04:59.265455 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264093    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 14:04:59.265455 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264096    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 14:04:59.265455 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264099    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 14:04:59.265455 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264101    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 14:04:59.265455 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264104    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 14:04:59.265455 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264107    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 14:04:59.265455 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264110    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 14:04:59.265455 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264112    2582 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 14:04:59.265455 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264115    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 14:04:59.265455 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264118    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 14:04:59.265455 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264121    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 14:04:59.265455 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264124    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 14:04:59.265455 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264126    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 14:04:59.266083 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264129    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 14:04:59.266083 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264131    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 14:04:59.266083 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264134    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 14:04:59.266083 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264137    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 14:04:59.266083 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264139    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 14:04:59.266083 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264142    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 14:04:59.266083 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264145    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 14:04:59.266083 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264147    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 14:04:59.266083 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264150    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 14:04:59.266083 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264152    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 14:04:59.266083 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264155    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 14:04:59.266083 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264158    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 14:04:59.266083 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264160    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 14:04:59.266083 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264163    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 14:04:59.266083 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264165    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 14:04:59.266083 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264168    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 14:04:59.266083 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264171    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 14:04:59.266083 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264174    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 14:04:59.266083 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264910    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 14:04:59.266563 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264918    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 14:04:59.266563 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264921    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 14:04:59.266563 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264924    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 14:04:59.266563 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264928    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 14:04:59.266563 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264931    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 14:04:59.266563 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264933    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 14:04:59.266563 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264937    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 14:04:59.266563 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264940    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 14:04:59.266563 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264942    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 14:04:59.266563 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264945    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 14:04:59.266563 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264948    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 14:04:59.266563 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264951    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 14:04:59.266563 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264954    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 14:04:59.266563 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264956    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 14:04:59.266563 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264959    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 14:04:59.266563 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264961    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 14:04:59.266563 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264965    2582 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 14:04:59.266563 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264968    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 14:04:59.266563 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264971    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 14:04:59.266563 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264974    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 14:04:59.267120 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264976    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 14:04:59.267120 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264979    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 14:04:59.267120 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264982    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 14:04:59.267120 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264985    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 14:04:59.267120 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264988    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 14:04:59.267120 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264990    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 14:04:59.267120 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264993    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 14:04:59.267120 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264996    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 14:04:59.267120 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.264999    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 14:04:59.267120 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265001    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 14:04:59.267120 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265004    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 14:04:59.267120 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265008    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 14:04:59.267120 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265013    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 14:04:59.267120 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265016    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 14:04:59.267120 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265019    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 14:04:59.267120 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265022    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 14:04:59.267120 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265025    2582 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 14:04:59.267120 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265027    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 14:04:59.267120 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265030    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 14:04:59.267589 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265033    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 14:04:59.267589 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265036    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 14:04:59.267589 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265039    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 14:04:59.267589 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265042    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 14:04:59.267589 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265045    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 14:04:59.267589 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265048    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 14:04:59.267589 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265050    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 14:04:59.267589 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265053    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 14:04:59.267589 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265055    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 14:04:59.267589 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265058    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 14:04:59.267589 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265061    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 14:04:59.267589 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265063    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 14:04:59.267589 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265066    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 14:04:59.267589 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265068    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 14:04:59.267589 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265073    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 14:04:59.267589 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265076    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 14:04:59.267589 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265079    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 14:04:59.267589 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265082    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 14:04:59.267589 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265085    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 14:04:59.267589 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265088    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 14:04:59.268092 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265091    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 14:04:59.268092 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265095    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 14:04:59.268092 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265097    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 14:04:59.268092 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265101    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 14:04:59.268092 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265104    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 14:04:59.268092 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265107    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 14:04:59.268092 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265110    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 14:04:59.268092 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265113    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 14:04:59.268092 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265117    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 14:04:59.268092 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265120    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 14:04:59.268092 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265122    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 14:04:59.268092 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265125    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 14:04:59.268092 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265128    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 14:04:59.268092 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265132    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 14:04:59.268092 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265135    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 14:04:59.268092 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265138    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 14:04:59.268092 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265141    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 14:04:59.268092 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265143    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 14:04:59.268092 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265146    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 14:04:59.268092 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265149    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 14:04:59.268568 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265151    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 14:04:59.268568 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265154    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 14:04:59.268568 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265156    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 14:04:59.268568 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265159    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 14:04:59.268568 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265161    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 14:04:59.268568 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265164    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 14:04:59.268568 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265167    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 14:04:59.268568 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265169    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 14:04:59.268568 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265172    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 14:04:59.268568 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265174    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 14:04:59.268568 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265177    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 14:04:59.268568 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265181    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 14:04:59.268568 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265183    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 14:04:59.268568 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265186    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 14:04:59.268568 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265189    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 14:04:59.268568 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265191    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 14:04:59.268568 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265194    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 14:04:59.268568 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265196    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 14:04:59.268568 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265199    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 14:04:59.268568 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265576    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 14:04:59.269072 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265584    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 14:04:59.269072 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265588    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 14:04:59.269072 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265591    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 14:04:59.269072 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265593    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 14:04:59.269072 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265596    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 14:04:59.269072 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265599    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 14:04:59.269072 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265602    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 14:04:59.269072 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265618    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 14:04:59.269072 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265621    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 14:04:59.269072 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265623    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 14:04:59.269072 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265626    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 14:04:59.269072 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265628    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 14:04:59.269072 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265631    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 14:04:59.269072 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265634    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 14:04:59.269072 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265637    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 14:04:59.269072 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265639    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 14:04:59.269072 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265642    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 14:04:59.269072 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265645    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 14:04:59.269072 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265647    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 14:04:59.269542 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265650    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 14:04:59.269542 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265652    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 14:04:59.269542 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265655    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 14:04:59.269542 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265657    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 14:04:59.269542 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265660    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 14:04:59.269542 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265663    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 14:04:59.269542 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265665    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 14:04:59.269542 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265668    2582 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 14:04:59.269542 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265671    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 14:04:59.269542 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265673    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 14:04:59.269542 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265676    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 14:04:59.269542 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265678    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 14:04:59.269542 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265681    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 14:04:59.269542 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265685    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 14:04:59.269542 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265688    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 14:04:59.269542 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265691    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 14:04:59.269542 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265694    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 14:04:59.269542 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265698    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 14:04:59.269542 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265702    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 14:04:59.270013 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265705    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 14:04:59.270013 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265709    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 14:04:59.270013 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265713    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 14:04:59.270013 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265715    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 14:04:59.270013 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265718    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 14:04:59.270013 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265721    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 14:04:59.270013 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265725    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 14:04:59.270013 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265728    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 14:04:59.270013 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265731    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 14:04:59.270013 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265734    2582 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 14:04:59.270013 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265736    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 14:04:59.270013 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265739    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 14:04:59.270013 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265742    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 14:04:59.270013 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265745    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 14:04:59.270013 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265747    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 14:04:59.270013 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265750    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 14:04:59.270013 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265752    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 14:04:59.270013 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265755    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 14:04:59.270013 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265757    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 14:04:59.270013 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265760    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 14:04:59.270502 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265762    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 14:04:59.270502 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265765    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 14:04:59.270502 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265767    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 14:04:59.270502 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265770    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 14:04:59.270502 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265772    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 14:04:59.270502 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265775    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 14:04:59.270502 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265779    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 14:04:59.270502 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265782    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 14:04:59.270502 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265785    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 14:04:59.270502 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265787    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 14:04:59.270502 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265790    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 14:04:59.270502 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265792    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 14:04:59.270502 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265795    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 14:04:59.270502 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265798    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 14:04:59.270502 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265801    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 14:04:59.270502 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265804    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 14:04:59.270502 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265806    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 14:04:59.270502 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265811    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 14:04:59.270502 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265813    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 14:04:59.270998 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265816    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 14:04:59.270998 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265818    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 14:04:59.270998 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265821    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 14:04:59.270998 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265824    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 14:04:59.270998 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265826    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 14:04:59.270998 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265829    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 14:04:59.270998 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265832    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 14:04:59.270998 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265835    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 14:04:59.270998 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265837    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 14:04:59.270998 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265840    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 14:04:59.270998 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265843    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 14:04:59.270998 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265845    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 14:04:59.270998 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265848    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 14:04:59.270998 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265850    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 14:04:59.270998 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265853    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 14:04:59.270998 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265856    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 14:04:59.270998 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265858    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 14:04:59.270998 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265860    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 14:04:59.270998 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265863    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 14:04:59.270998 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265867    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 14:04:59.271486 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.265870    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 14:04:59.271486 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.265948    2582 flags.go:64] FLAG: --address="0.0.0.0"
Sep 17 14:04:59.271486 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.265982    2582 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 17 14:04:59.271486 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.265988    2582 flags.go:64] FLAG: --anonymous-auth="true"
Sep 17 14:04:59.271486 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.265993    2582 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 17 14:04:59.271486 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.265997    2582 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 17 14:04:59.271486 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266000    2582 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 17 14:04:59.271486 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266004    2582 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 17 14:04:59.271486 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266008    2582 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 17 14:04:59.271486 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266012    2582 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 17 14:04:59.271486 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266015    2582 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 17 14:04:59.271486 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266018    2582 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 17 14:04:59.271486 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266022    2582 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 17 14:04:59.271486 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266025    2582 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 17 14:04:59.271486 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266027    2582 flags.go:64] FLAG: --cgroup-root=""
Sep 17 14:04:59.271486 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266030    2582 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 17 14:04:59.271486 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266033    2582 flags.go:64] FLAG: --client-ca-file=""
Sep 17 14:04:59.271486 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266036    2582 flags.go:64] FLAG: --cloud-provider="external"
Sep 17 14:04:59.271486 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266039    2582 flags.go:64] FLAG: --cluster-dns="[]"
Sep 17 14:04:59.271486 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266043    2582 flags.go:64] FLAG: --cluster-domain=""
Sep 17 14:04:59.271486 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266046    2582 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 17 14:04:59.271486 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266049    2582 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 17 14:04:59.271486 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266052    2582 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 17 14:04:59.271486 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266055    2582 flags.go:64] FLAG: --container-log-max-files="5"
Sep 17 14:04:59.272117 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266059    2582 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 17 14:04:59.272117 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266062    2582 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 17 14:04:59.272117 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266065    2582 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 17 14:04:59.272117 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266068    2582 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 17 14:04:59.272117 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266072    2582 flags.go:64] FLAG: --contention-profiling="false"
Sep 17 14:04:59.272117 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266075    2582 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 17 14:04:59.272117 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266078    2582 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 17 14:04:59.272117 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266083    2582 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 17 14:04:59.272117 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266086    2582 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 17 14:04:59.272117 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266090    2582 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 17 14:04:59.272117 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266094    2582 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 17 14:04:59.272117 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266097    2582 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 17 14:04:59.272117 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266100    2582 flags.go:64] FLAG: --enable-load-reader="false"
Sep 17 14:04:59.272117 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266103    2582 flags.go:64] FLAG: --enable-server="true"
Sep 17 14:04:59.272117 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266106    2582 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 17 14:04:59.272117 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266111    2582 flags.go:64] FLAG: --event-burst="100"
Sep 17 14:04:59.272117 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266115    2582 flags.go:64] FLAG: --event-qps="50"
Sep 17 14:04:59.272117 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266118    2582 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 17 14:04:59.272117 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266121    2582 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 17 14:04:59.272117 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266124    2582 flags.go:64] FLAG: --eviction-hard=""
Sep 17 14:04:59.272117 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266128    2582 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 17 14:04:59.272117 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266131    2582 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 17 14:04:59.272117 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266134    2582 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 17 14:04:59.272117 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266137    2582 flags.go:64] FLAG: --eviction-soft=""
Sep 17 14:04:59.272117 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266140    2582 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 17 14:04:59.272735 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266143    2582 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 17 14:04:59.272735 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266146    2582 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 17 14:04:59.272735 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266149    2582 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 17 14:04:59.272735 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266152    2582 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 17 14:04:59.272735 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266155    2582 flags.go:64] FLAG: --fail-swap-on="true"
Sep 17 14:04:59.272735 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266158    2582 flags.go:64] FLAG: --feature-gates=""
Sep 17 14:04:59.272735 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266161    2582 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 17 14:04:59.272735 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266164    2582 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 17 14:04:59.272735 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266167    2582 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 17 14:04:59.272735 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266170    2582 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 17 14:04:59.272735 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266175    2582 flags.go:64] FLAG: --healthz-port="10248"
Sep 17 14:04:59.272735 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266178    2582 flags.go:64] FLAG: --help="false"
Sep 17 14:04:59.272735 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266181    2582 flags.go:64] FLAG: --hostname-override="ip-10-0-137-53.ec2.internal"
Sep 17 14:04:59.272735 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266184    2582 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 17 14:04:59.272735 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266189    2582 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 17 14:04:59.272735 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266192    2582 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 17 14:04:59.272735 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266195    2582 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 17 14:04:59.272735 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266198    2582 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 17 14:04:59.272735 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266201    2582 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 17 14:04:59.272735 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266204    2582 flags.go:64] FLAG: --image-service-endpoint=""
Sep 17 14:04:59.272735 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266207    2582 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 17 14:04:59.272735 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266210    2582 flags.go:64] FLAG: --kube-api-burst="100"
Sep 17 14:04:59.272735 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266213    2582 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 17 14:04:59.273467 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266217    2582 flags.go:64] FLAG: --kube-api-qps="50"
Sep 17 14:04:59.273467 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266220    2582 flags.go:64] FLAG: --kube-reserved=""
Sep 17 14:04:59.273467 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266223    2582 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 17 14:04:59.273467 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266226    2582 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 17 14:04:59.273467 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266229    2582 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 17 14:04:59.273467 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266232    2582 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 17 14:04:59.273467 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266235    2582 flags.go:64] FLAG: --lock-file=""
Sep 17 14:04:59.273467 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266237    2582 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 17 14:04:59.273467 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266240    2582 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 17 14:04:59.273467 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266243    2582 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 17 14:04:59.273467 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266248    2582 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 17 14:04:59.273467 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266251    2582 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 17 14:04:59.273467 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266254    2582 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 17 14:04:59.273467 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266257    2582 flags.go:64] FLAG: --logging-format="text"
Sep 17 14:04:59.273467 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266260    2582 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 17 14:04:59.273467 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266263    2582 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 17 14:04:59.273467 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266266    2582 flags.go:64] FLAG: --manifest-url=""
Sep 17 14:04:59.273467 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266268    2582 flags.go:64] FLAG: --manifest-url-header=""
Sep 17 14:04:59.273467 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266272    2582 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 17 14:04:59.273467 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266276    2582 flags.go:64] FLAG: --max-open-files="1000000"
Sep 17 14:04:59.273467 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266287    2582 flags.go:64] FLAG: --max-pods="110"
Sep 17 14:04:59.273467 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266290    2582 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 17 14:04:59.273467 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266294    2582 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 17 14:04:59.273467 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266298    2582 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 17 14:04:59.273467 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266301    2582 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 17 14:04:59.274468 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266304    2582 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 17 14:04:59.274468 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266307    2582 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 17 14:04:59.274468 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266310    2582 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 17 14:04:59.274468 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266317    2582 flags.go:64] FLAG: --node-status-max-images="50"
Sep 17 14:04:59.274468 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266321    2582 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 17 14:04:59.274468 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266324    2582 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 17 14:04:59.274468 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266327    2582 flags.go:64] FLAG: --pod-cidr=""
Sep 17 14:04:59.274468 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266330    2582 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 17 14:04:59.274468 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266335    2582 flags.go:64] FLAG: --pod-manifest-path=""
Sep 17 14:04:59.274468 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266338    2582 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 17 14:04:59.274468 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266342    2582 flags.go:64] FLAG: --pods-per-core="0"
Sep 17 14:04:59.274468 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266345    2582 flags.go:64] FLAG: --port="10250"
Sep 17 14:04:59.274468 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266348    2582 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 17 14:04:59.274468 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266351    2582 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-022e905750458817d"
Sep 17 14:04:59.274468 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266354    2582 flags.go:64] FLAG: --qos-reserved=""
Sep 17 14:04:59.274468 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266357    2582 flags.go:64] FLAG: --read-only-port="10255"
Sep 17 14:04:59.274468 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266360    2582 flags.go:64] FLAG: --register-node="true"
Sep 17 14:04:59.274468 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266363    2582 flags.go:64] FLAG: --register-with-taints=""
Sep 17 14:04:59.274468 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266367    2582 flags.go:64] FLAG: --registry-burst="10"
Sep 17 14:04:59.274468 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266370    2582 flags.go:64] FLAG: --registry-qps="5"
Sep 17 14:04:59.274468 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266373    2582 flags.go:64] FLAG: --reserved-cpus=""
Sep 17 14:04:59.274468 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266376    2582 flags.go:64] FLAG: --reserved-memory=""
Sep 17 14:04:59.274468 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266379    2582 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 17 14:04:59.274468 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266382    2582 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 17 14:04:59.274468 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266385    2582 flags.go:64] FLAG: --rotate-certificates="false"
Sep 17 14:04:59.275130 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266388    2582 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 17 14:04:59.275130 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266391    2582 flags.go:64] FLAG: --runonce="false"
Sep 17 14:04:59.275130 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266395    2582 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 17 14:04:59.275130 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266398    2582 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 17 14:04:59.275130 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266401    2582 flags.go:64] FLAG: --seccomp-default="false"
Sep 17 14:04:59.275130 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266405    2582 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 17 14:04:59.275130 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266408    2582 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 17 14:04:59.275130 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266411    2582 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 17 14:04:59.275130 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266414    2582 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 17 14:04:59.275130 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266417    2582 flags.go:64] FLAG: --storage-driver-password="root"
Sep 17 14:04:59.275130 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266420    2582 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 17 14:04:59.275130 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266422    2582 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 17 14:04:59.275130 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266425    2582 flags.go:64] FLAG: --storage-driver-user="root"
Sep 17 14:04:59.275130 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266428    2582 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 17 14:04:59.275130 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266431    2582 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 17 14:04:59.275130 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266434    2582 flags.go:64] FLAG: --system-cgroups=""
Sep 17 14:04:59.275130 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266437    2582 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 17 14:04:59.275130 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266445    2582 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 17 14:04:59.275130 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266448    2582 flags.go:64] FLAG: --tls-cert-file=""
Sep 17 14:04:59.275130 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266451    2582 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 17 14:04:59.275130 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266455    2582 flags.go:64] FLAG: --tls-min-version=""
Sep 17 14:04:59.275130 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266458    2582 flags.go:64] FLAG: --tls-private-key-file=""
Sep 17 14:04:59.275130 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266460    2582 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 17 14:04:59.275130 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266463    2582 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 17 14:04:59.275130 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266466    2582 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 17 14:04:59.275757 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266469    2582 flags.go:64] FLAG: --v="2"
Sep 17 14:04:59.275757 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266473    2582 flags.go:64] FLAG: --version="false"
Sep 17 14:04:59.275757 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266476    2582 flags.go:64] FLAG: --vmodule=""
Sep 17 14:04:59.275757 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266480    2582 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 17 14:04:59.275757 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.266483    2582 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 17 14:04:59.275757 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266626    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 14:04:59.275757 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266630    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 14:04:59.275757 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266634    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 14:04:59.275757 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266638    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 14:04:59.275757 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266642    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 14:04:59.275757 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266646    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 14:04:59.275757 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266649    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 14:04:59.275757 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266652    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 14:04:59.275757 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266655    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 14:04:59.275757 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266657    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 14:04:59.275757 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266660    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 14:04:59.275757 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266663    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 14:04:59.275757 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266666    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 14:04:59.275757 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266668    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 14:04:59.275757 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266671    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 14:04:59.275757 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266674    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 14:04:59.276319 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266677    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 14:04:59.276319 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266680    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 14:04:59.276319 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266684    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 14:04:59.276319 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266687    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 14:04:59.276319 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266690    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 14:04:59.276319 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266693    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 14:04:59.276319 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266695    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 14:04:59.276319 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266698    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 14:04:59.276319 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266701    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 14:04:59.276319 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266703    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 14:04:59.276319 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266706    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 14:04:59.276319 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266709    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 14:04:59.276319 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266712    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 14:04:59.276319 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266714    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 14:04:59.276319 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266717    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 14:04:59.276319 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266720    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 14:04:59.276319 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266723    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 14:04:59.276319 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266725    2582 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 14:04:59.276319 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266728    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 14:04:59.276319 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266731    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 14:04:59.276834 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266735    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 14:04:59.276834 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266738    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 14:04:59.276834 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266741    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 14:04:59.276834 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266744    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 14:04:59.276834 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266746    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 14:04:59.276834 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266749    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 14:04:59.276834 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266752    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 14:04:59.276834 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266754    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 14:04:59.276834 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266757    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 14:04:59.276834 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266760    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 14:04:59.276834 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266762    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 14:04:59.276834 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266765    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 14:04:59.276834 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266767    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 14:04:59.276834 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266770    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 14:04:59.276834 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266773    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 14:04:59.276834 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266776    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 14:04:59.276834 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266779    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 14:04:59.276834 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266782    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 14:04:59.276834 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266784    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 14:04:59.276834 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266787    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 14:04:59.277342 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266790    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 14:04:59.277342 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266792    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 14:04:59.277342 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266795    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 14:04:59.277342 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266798    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 14:04:59.277342 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266800    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 14:04:59.277342 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266803    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 14:04:59.277342 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266805    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 14:04:59.277342 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266808    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 14:04:59.277342 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266811    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 14:04:59.277342 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266813    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 14:04:59.277342 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266816    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 14:04:59.277342 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266820    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 14:04:59.277342 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266825    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 14:04:59.277342 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266828    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 14:04:59.277342 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266831    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 14:04:59.277342 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266834    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 14:04:59.277342 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266837    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 14:04:59.277342 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266840    2582 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 14:04:59.277342 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266842    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 14:04:59.277825 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266845    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 14:04:59.277825 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266848    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 14:04:59.277825 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266851    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 14:04:59.277825 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266853    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 14:04:59.277825 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266856    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 14:04:59.277825 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266859    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 14:04:59.277825 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266861    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 14:04:59.277825 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266864    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 14:04:59.277825 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266867    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 14:04:59.277825 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266869    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 14:04:59.277825 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266872    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 14:04:59.277825 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266875    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 14:04:59.277825 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266878    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 14:04:59.277825 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266881    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 14:04:59.277825 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266883    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 14:04:59.277825 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266886    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 14:04:59.277825 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266888    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 14:04:59.277825 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266891    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 14:04:59.277825 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266895    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 14:04:59.278310 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266898    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 14:04:59.278310 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266901    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 14:04:59.278310 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266904    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 14:04:59.278310 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266907    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 14:04:59.278310 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.266909    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 14:04:59.278310 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.267467    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 14:04:59.278310 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.275976    2582 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 17 14:04:59.278310 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.275994    2582 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 17 14:04:59.278310 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.276774    2582 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 17 14:04:59.278310 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.276783    2582 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 17 14:04:59.278310 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276848    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 14:04:59.278310 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276854    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 14:04:59.278310 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276857    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 14:04:59.278310 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276862    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 14:04:59.278684 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276866    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 14:04:59.278684 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276869    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 14:04:59.278684 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276873    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 14:04:59.278684 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276876    2582 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 14:04:59.278684 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276879    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 14:04:59.278684 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276882    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 14:04:59.278684 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276884    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 14:04:59.278684 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276887    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 14:04:59.278684 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276890    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 14:04:59.278684 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276894    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 14:04:59.278684 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276898    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 14:04:59.278684 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276902    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 14:04:59.278684 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276906    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 14:04:59.278684 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276909    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 14:04:59.278684 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276912    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 14:04:59.278684 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276915    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 14:04:59.278684 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276918    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 14:04:59.278684 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276920    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 14:04:59.278684 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276923    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 14:04:59.278684 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276926    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 14:04:59.279171 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276928    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 14:04:59.279171 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276932    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 14:04:59.279171 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276934    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 14:04:59.279171 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276937    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 14:04:59.279171 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276939    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 14:04:59.279171 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276943    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 14:04:59.279171 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276945    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 14:04:59.279171 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276948    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 14:04:59.279171 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276951    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 14:04:59.279171 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276953    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 14:04:59.279171 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276956    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 14:04:59.279171 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276959    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 14:04:59.279171 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276961    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 14:04:59.279171 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276964    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 14:04:59.279171 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276966    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 14:04:59.279171 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276969    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 14:04:59.279171 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276972    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 14:04:59.279171 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276975    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 14:04:59.279171 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276977    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 14:04:59.279171 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276980    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 14:04:59.279761 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276982    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 14:04:59.279761 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276985    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 14:04:59.279761 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276988    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 14:04:59.279761 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276990    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 14:04:59.279761 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276993    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 14:04:59.279761 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276996    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 14:04:59.279761 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.276998    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 14:04:59.279761 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277001    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 14:04:59.279761 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277004    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 14:04:59.279761 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277006    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 14:04:59.279761 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277009    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 14:04:59.279761 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277012    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 14:04:59.279761 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277014    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 14:04:59.279761 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277017    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 14:04:59.279761 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277019    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 14:04:59.279761 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277022    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 14:04:59.279761 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277024    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 14:04:59.279761 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277028    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 14:04:59.279761 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277031    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 14:04:59.279761 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277034    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 14:04:59.280250 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277037    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 14:04:59.280250 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277039    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 14:04:59.280250 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277042    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 14:04:59.280250 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277044    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 14:04:59.280250 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277047    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 14:04:59.280250 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277050    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 14:04:59.280250 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277052    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 14:04:59.280250 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277055    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 14:04:59.280250 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277057    2582 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 14:04:59.280250 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277060    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 14:04:59.280250 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277063    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 14:04:59.280250 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277065    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 14:04:59.280250 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277068    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 14:04:59.280250 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277071    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 14:04:59.280250 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277073    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 14:04:59.280250 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277077    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 14:04:59.280250 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277080    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 14:04:59.280250 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277082    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 14:04:59.280250 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277085    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 14:04:59.280250 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277087    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 14:04:59.280748 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277090    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 14:04:59.280748 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277092    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 14:04:59.280748 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277095    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 14:04:59.280748 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277098    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 14:04:59.280748 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277100    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 14:04:59.280748 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277103    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 14:04:59.280748 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277105    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 14:04:59.280748 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277108    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 14:04:59.280748 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277111    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 14:04:59.280748 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277114    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 14:04:59.280748 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277117    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 14:04:59.280748 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277120    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 14:04:59.280748 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277123    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 14:04:59.280748 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277125    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 14:04:59.280748 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277128    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 14:04:59.281118 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.277133    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 14:04:59.281118 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277260    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 14:04:59.281118 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277267    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 14:04:59.281118 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277271    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 14:04:59.281118 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277275    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 14:04:59.281118 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277278    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 14:04:59.281118 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277281    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 14:04:59.281118 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277284    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 14:04:59.281118 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277287    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 14:04:59.281118 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277289    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 14:04:59.281118 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277292    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 14:04:59.281118 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277296    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 14:04:59.281118 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277300    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 14:04:59.281118 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277303    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 14:04:59.281476 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277307    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 14:04:59.281476 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277310    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 14:04:59.281476 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277312    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 14:04:59.281476 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277315    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 14:04:59.281476 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277318    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 14:04:59.281476 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277321    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 14:04:59.281476 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277324    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 14:04:59.281476 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277327    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 14:04:59.281476 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277330    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 14:04:59.281476 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277333    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 14:04:59.281476 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277335    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 14:04:59.281476 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277338    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 14:04:59.281476 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277341    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 14:04:59.281476 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277344    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 14:04:59.281476 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277347    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 14:04:59.281476 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277349    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 14:04:59.281476 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277353    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 14:04:59.281476 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277355    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 14:04:59.281476 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277358    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 14:04:59.281955 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277361    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 14:04:59.281955 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277363    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 14:04:59.281955 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277366    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 14:04:59.281955 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277368    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 14:04:59.281955 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277371    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 14:04:59.281955 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277373    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 14:04:59.281955 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277376    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 14:04:59.281955 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277378    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 14:04:59.281955 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277381    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 14:04:59.281955 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277384    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 14:04:59.281955 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277386    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 14:04:59.281955 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277389    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 14:04:59.281955 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277391    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 14:04:59.281955 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277395    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 14:04:59.281955 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277397    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 14:04:59.281955 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277400    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 14:04:59.281955 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277403    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 14:04:59.281955 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277405    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 14:04:59.281955 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277408    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 14:04:59.281955 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277411    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 14:04:59.282451 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277414    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 14:04:59.282451 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277417    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 14:04:59.282451 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277419    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 14:04:59.282451 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277422    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 14:04:59.282451 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277425    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 14:04:59.282451 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277427    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 14:04:59.282451 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277430    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 14:04:59.282451 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277433    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 14:04:59.282451 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277435    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 14:04:59.282451 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277438    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 14:04:59.282451 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277440    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 14:04:59.282451 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277442    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 14:04:59.282451 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277445    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 14:04:59.282451 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277447    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 14:04:59.282451 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277450    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 14:04:59.282451 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277452    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 14:04:59.282451 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277455    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 14:04:59.282451 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277458    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 14:04:59.282451 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277460    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 14:04:59.282451 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277462    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 14:04:59.282955 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277465    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 14:04:59.282955 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277468    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 14:04:59.282955 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277470    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 14:04:59.282955 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277473    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 14:04:59.282955 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277476    2582 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 14:04:59.282955 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277478    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 14:04:59.282955 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277481    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 14:04:59.282955 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277484    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 14:04:59.282955 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277487    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 14:04:59.282955 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277489    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 14:04:59.282955 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277492    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 14:04:59.282955 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277495    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 14:04:59.282955 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277497    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 14:04:59.282955 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277500    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 14:04:59.282955 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277504    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 14:04:59.282955 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277507    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 14:04:59.282955 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277510    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 14:04:59.282955 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277513    2582 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 14:04:59.282955 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277516    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 14:04:59.282955 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277518    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 14:04:59.282955 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277521    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 14:04:59.283470 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277523    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 14:04:59.283470 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277526    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 14:04:59.283470 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277529    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 14:04:59.283470 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277531    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 14:04:59.283470 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277534    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 14:04:59.283470 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:04:59.277536    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 14:04:59.283470 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.277541    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 14:04:59.283470 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.277661    2582 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 17 14:04:59.283470 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.280414    2582 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 17 14:04:59.283470 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.281347    2582 server.go:1019] "Starting client certificate rotation"
Sep 17 14:04:59.283470 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.281453    2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 17 14:04:59.283470 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.281487    2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 17 14:04:59.284013 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.283999    2582 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 17 14:04:59.285189 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.285161    2582 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 17 14:04:59.299091 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.299071    2582 log.go:25] "Validated CRI v1 runtime API"
Sep 17 14:04:59.305825 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.305809    2582 log.go:25] "Validated CRI v1 image API"
Sep 17 14:04:59.306674 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.306653    2582 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 17 14:04:59.309778 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.309759    2582 fs.go:135] Filesystem UUIDs: map[775716aa-92d0-4226-b055-5eb5bc9d955e:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 c89f488d-9cfe-488a-b8e9-a72748a8b966:/dev/nvme0n1p3]
Sep 17 14:04:59.309832 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.309778    2582 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 17 14:04:59.316288 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.316026    2582 manager.go:216] Machine: {Timestamp:2026-09-17 14:04:59.314581523 +0000 UTC m=+0.316124989 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100515 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec261cff1ea79304fdaded6294acbf12 SystemUUID:ec261cff-1ea7-9304-fdad-ed6294acbf12 BootID:9d0cfabb-3ece-4594-af22-90de5390f78b Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:dc:0f:e5:65 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:dc:0f:e5:65 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:52:76:18:0e:a0:5a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 17 14:04:59.316288 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.316277    2582 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 17 14:04:59.316429 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.316365    2582 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 17 14:04:59.319627 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.319588    2582 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 17 14:04:59.319765 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.319630    2582 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-53.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 17 14:04:59.320372 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.320354    2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 17 14:04:59.320477 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.320465    2582 topology_manager.go:138] "Creating topology manager with none policy"
Sep 17 14:04:59.320514 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.320479    2582 container_manager_linux.go:309] "Creating device plugin manager"
Sep 17 14:04:59.320514 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.320488    2582 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 17 14:04:59.321250 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.321239    2582 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 17 14:04:59.322348 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.322338    2582 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 17 14:04:59.326039 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.326028    2582 state_mem.go:36] "Initialized new in-memory state store"
Sep 17 14:04:59.326147 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.326138    2582 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 17 14:04:59.326183 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.326167    2582 kubelet.go:486] "Attempting to sync node with API server"
Sep 17 14:04:59.326183 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.326176    2582 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 17 14:04:59.326249 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.326191    2582 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 17 14:04:59.326249 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.326200    2582 kubelet.go:398] "Adding apiserver pod source"
Sep 17 14:04:59.326249 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.326209    2582 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 17 14:04:59.326592 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.326581    2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 17 14:04:59.326645 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.326623    2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 17 14:04:59.329029 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.329012    2582 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 17 14:04:59.331384 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.331369    2582 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 17 14:04:59.331442 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.331400    2582 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 17 14:04:59.333446 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.333433    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 17 14:04:59.333509 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.333450    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 17 14:04:59.333509 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.333456    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 17 14:04:59.333509 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.333461    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 17 14:04:59.333509 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.333470    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 17 14:04:59.333509 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.333476    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 17 14:04:59.333509 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.333489    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 17 14:04:59.333509 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.333494    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 17 14:04:59.333509 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.333502    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 17 14:04:59.333509 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.333508    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 17 14:04:59.333824 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.333522    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 17 14:04:59.333824 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.333528    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 17 14:04:59.334373 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.334361    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 17 14:04:59.334373 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.334369    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 17 14:04:59.338173 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.338160    2582 server.go:1268] "Started kubelet"
Sep 17 14:04:59.338259 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.338219    2582 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 17 14:04:59.338312 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.338248    2582 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 17 14:04:59.338353 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.338337    2582 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 17 14:04:59.338980 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.338961    2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-53.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 17 14:04:59.339158 ip-10-0-137-53 systemd[1]: Started Kubernetes Kubelet.
Sep 17 14:04:59.340172 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:04:59.340077    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 17 14:04:59.340172 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:04:59.340091    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-53.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 17 14:04:59.340620 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.340594    2582 server.go:310] "Adding debug handlers to kubelet server"
Sep 17 14:04:59.349330 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.349306    2582 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 17 14:04:59.350278 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:04:59.350250    2582 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 17 14:04:59.353569 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.353542    2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 17 14:04:59.354219 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.354203    2582 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 17 14:04:59.354329 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:04:59.354308    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-53.ec2.internal\" not found"
Sep 17 14:04:59.354955 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.354939    2582 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 17 14:04:59.355052 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.354938    2582 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 17 14:04:59.355052 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.354973    2582 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 17 14:04:59.355160 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.355080    2582 reconstruct.go:97] "Volume reconstruction finished"
Sep 17 14:04:59.355160 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.355091    2582 reconciler.go:29] "Reconciler: start to sync state"
Sep 17 14:04:59.355160 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.355135    2582 factory.go:55] Registering systemd factory
Sep 17 14:04:59.355278 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.355172    2582 factory.go:223] Registration of the systemd container factory successfully
Sep 17 14:04:59.355384 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.355366    2582 factory.go:153] Registering CRI-O factory
Sep 17 14:04:59.355384 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.355386    2582 factory.go:223] Registration of the crio container factory successfully
Sep 17 14:04:59.355525 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.355433    2582 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 17 14:04:59.355525 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.355450    2582 factory.go:103] Registering Raw factory
Sep 17 14:04:59.355525 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.355464    2582 manager.go:1196] Started watching for new ooms in manager
Sep 17 14:04:59.355825 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.355812    2582 manager.go:349] Starting recovery of all containers
Sep 17 14:04:59.356228 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:04:59.355121    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-53.ec2.internal.18d620bd4cf136ce  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-53.ec2.internal,UID:ip-10-0-137-53.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-53.ec2.internal,},FirstTimestamp:2026-09-17 14:04:59.338135246 +0000 UTC m=+0.339678712,LastTimestamp:2026-09-17 14:04:59.338135246 +0000 UTC m=+0.339678712,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-53.ec2.internal,}"
Sep 17 14:04:59.360949 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:04:59.360922    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 17 14:04:59.361067 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:04:59.361020    2582 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-53.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 17 14:04:59.365477 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.365462    2582 manager.go:354] Recovery completed
Sep 17 14:04:59.369835 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.369823    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 14:04:59.371493 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.371467    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-53.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 14:04:59.371581 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.371503    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-53.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 14:04:59.371581 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.371512    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-53.ec2.internal" event="NodeHasSufficientPID"
Sep 17 14:04:59.372063 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.372051    2582 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 17 14:04:59.372063 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.372062    2582 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 17 14:04:59.372157 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.372076    2582 state_mem.go:36] "Initialized new in-memory state store"
Sep 17 14:04:59.372157 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.372107    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 14:04:59.372776 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.372759    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-53.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 14:04:59.372838 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.372789    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-53.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 14:04:59.372838 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.372802    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-53.ec2.internal" event="NodeHasSufficientPID"
Sep 17 14:04:59.373820 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:04:59.373753    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-53.ec2.internal.18d620bd4eee2ea7  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-53.ec2.internal,UID:ip-10-0-137-53.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-53.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-53.ec2.internal,},FirstTimestamp:2026-09-17 14:04:59.371490983 +0000 UTC m=+0.373034451,LastTimestamp:2026-09-17 14:04:59.371490983 +0000 UTC m=+0.373034451,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-53.ec2.internal,}"
Sep 17 14:04:59.382984 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:04:59.382896    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-53.ec2.internal.18d620bd4eee6f71  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-53.ec2.internal,UID:ip-10-0-137-53.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-137-53.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-137-53.ec2.internal,},FirstTimestamp:2026-09-17 14:04:59.371507569 +0000 UTC m=+0.373051037,LastTimestamp:2026-09-17 14:04:59.371507569 +0000 UTC m=+0.373051037,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-53.ec2.internal,}"
Sep 17 14:04:59.394161 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:04:59.394097    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-53.ec2.internal.18d620bd4eee91d5  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-53.ec2.internal,UID:ip-10-0-137-53.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-137-53.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-137-53.ec2.internal,},FirstTimestamp:2026-09-17 14:04:59.371516373 +0000 UTC m=+0.373059837,LastTimestamp:2026-09-17 14:04:59.371516373 +0000 UTC m=+0.373059837,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-53.ec2.internal,}"
Sep 17 14:04:59.511023 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:04:59.416177    2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-137-53.ec2.internal.18d620bd4eee2ea7\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-53.ec2.internal.18d620bd4eee2ea7  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-53.ec2.internal,UID:ip-10-0-137-53.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-53.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-53.ec2.internal,},FirstTimestamp:2026-09-17 14:04:59.371490983 +0000 UTC m=+0.373034451,LastTimestamp:2026-09-17 14:04:59.372774299 +0000 UTC m=+0.374317766,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-53.ec2.internal,}"
Sep 17 14:04:59.511023 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:04:59.424756    2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-137-53.ec2.internal.18d620bd4eee6f71\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-53.ec2.internal.18d620bd4eee6f71  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-53.ec2.internal,UID:ip-10-0-137-53.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-137-53.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-137-53.ec2.internal,},FirstTimestamp:2026-09-17 14:04:59.371507569 +0000 UTC m=+0.373051037,LastTimestamp:2026-09-17 14:04:59.372795298 +0000 UTC m=+0.374338768,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-53.ec2.internal,}"
Sep 17 14:04:59.511023 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:04:59.439092    2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-137-53.ec2.internal.18d620bd4eee91d5\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-53.ec2.internal.18d620bd4eee91d5  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-53.ec2.internal,UID:ip-10-0-137-53.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-137-53.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-137-53.ec2.internal,},FirstTimestamp:2026-09-17 14:04:59.371516373 +0000 UTC m=+0.373059837,LastTimestamp:2026-09-17 14:04:59.372807809 +0000 UTC m=+0.374351276,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-53.ec2.internal,}"
Sep 17 14:04:59.511023 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:04:59.455233    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-53.ec2.internal\" not found"
Sep 17 14:04:59.511023 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.485329    2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 17 14:04:59.511023 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.486525    2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 17 14:04:59.511023 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.486569    2582 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 17 14:04:59.511023 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.486591    2582 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 17 14:04:59.511023 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:04:59.486650    2582 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Sep 17 14:04:59.511388 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.511276    2582 policy_none.go:49] "None policy: Start"
Sep 17 14:04:59.511388 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.511293    2582 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 17 14:04:59.511388 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.511305    2582 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 17 14:04:59.512308 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.512296    2582 policy_none.go:47] "Start"
Sep 17 14:04:59.516836 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:04:59.516813    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Sep 17 14:04:59.520389 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.520373    2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-76hbt"
Sep 17 14:04:59.542312 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.542295    2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-76hbt"
Sep 17 14:04:59.554193 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.554174    2582 manager.go:337] "Starting Device Plugin manager"
Sep 17 14:04:59.554269 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:04:59.554209    2582 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 17 14:04:59.554269 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.554218    2582 server.go:87] "Starting device plugin registration server"
Sep 17 14:04:59.554565 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.554554    2582 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 17 14:04:59.554618 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.554567    2582 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 17 14:04:59.554712 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.554688    2582 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 17 14:04:59.554819 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.554772    2582 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 17 14:04:59.554819 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.554781    2582 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 17 14:04:59.555285 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:04:59.555263    2582 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 17 14:04:59.555377 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:04:59.555320    2582 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-53.ec2.internal\" not found"
Sep 17 14:04:59.571082 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:04:59.571061    2582 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-137-53.ec2.internal\" not found" node="ip-10-0-137-53.ec2.internal"
Sep 17 14:04:59.587293 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.587273    2582 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-53.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-53.ec2.internal"]
Sep 17 14:04:59.587360 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.587337    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 14:04:59.588775 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.588762    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-53.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 14:04:59.588846 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.588788    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-53.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 14:04:59.588846 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.588799    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-53.ec2.internal" event="NodeHasSufficientPID"
Sep 17 14:04:59.589939 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.589926    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 14:04:59.590070 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.590056    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-53.ec2.internal"
Sep 17 14:04:59.590140 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.590091    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 14:04:59.590619 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.590590    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-53.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 14:04:59.590696 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.590640    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-53.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 14:04:59.590696 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.590656    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-53.ec2.internal" event="NodeHasSufficientPID"
Sep 17 14:04:59.590774 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.590702    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-53.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 14:04:59.590774 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.590738    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-53.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 14:04:59.590774 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.590753    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-53.ec2.internal" event="NodeHasSufficientPID"
Sep 17 14:04:59.591911 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.591896    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-53.ec2.internal"
Sep 17 14:04:59.591961 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.591929    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 14:04:59.592578 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.592564    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-53.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 14:04:59.592660 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.592585    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-53.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 14:04:59.592660 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.592595    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-53.ec2.internal" event="NodeHasSufficientPID"
Sep 17 14:04:59.607144 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:04:59.607127    2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-53.ec2.internal\" not found" node="ip-10-0-137-53.ec2.internal"
Sep 17 14:04:59.610517 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:04:59.610504    2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-53.ec2.internal\" not found" node="ip-10-0-137-53.ec2.internal"
Sep 17 14:04:59.654696 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.654675    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 14:04:59.655327 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.655302    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ed4e451484feca11635366f26cff0482-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-53.ec2.internal\" (UID: \"ed4e451484feca11635366f26cff0482\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-53.ec2.internal"
Sep 17 14:04:59.655414 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.655328    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3ed51660b0e0cea0ccab4eff146b4e99-config\") pod \"kube-apiserver-proxy-ip-10-0-137-53.ec2.internal\" (UID: \"3ed51660b0e0cea0ccab4eff146b4e99\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-53.ec2.internal"
Sep 17 14:04:59.655414 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.655350    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ed4e451484feca11635366f26cff0482-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-53.ec2.internal\" (UID: \"ed4e451484feca11635366f26cff0482\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-53.ec2.internal"
Sep 17 14:04:59.655496 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.655417    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-53.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 14:04:59.655496 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.655442    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-53.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 14:04:59.655496 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.655471    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-53.ec2.internal" event="NodeHasSufficientPID"
Sep 17 14:04:59.655496 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.655494    2582 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-53.ec2.internal"
Sep 17 14:04:59.664569 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.664555    2582 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-53.ec2.internal"
Sep 17 14:04:59.664648 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:04:59.664575    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-53.ec2.internal\": node \"ip-10-0-137-53.ec2.internal\" not found"
Sep 17 14:04:59.712433 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:04:59.712400    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-53.ec2.internal\" not found"
Sep 17 14:04:59.756485 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.756443    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ed4e451484feca11635366f26cff0482-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-53.ec2.internal\" (UID: \"ed4e451484feca11635366f26cff0482\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-53.ec2.internal"
Sep 17 14:04:59.756485 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.756483    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ed4e451484feca11635366f26cff0482-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-53.ec2.internal\" (UID: \"ed4e451484feca11635366f26cff0482\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-53.ec2.internal"
Sep 17 14:04:59.756485 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.756500    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3ed51660b0e0cea0ccab4eff146b4e99-config\") pod \"kube-apiserver-proxy-ip-10-0-137-53.ec2.internal\" (UID: \"3ed51660b0e0cea0ccab4eff146b4e99\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-53.ec2.internal"
Sep 17 14:04:59.756767 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.756594    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ed4e451484feca11635366f26cff0482-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-53.ec2.internal\" (UID: \"ed4e451484feca11635366f26cff0482\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-53.ec2.internal"
Sep 17 14:04:59.756767 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.756603    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3ed51660b0e0cea0ccab4eff146b4e99-config\") pod \"kube-apiserver-proxy-ip-10-0-137-53.ec2.internal\" (UID: \"3ed51660b0e0cea0ccab4eff146b4e99\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-53.ec2.internal"
Sep 17 14:04:59.756767 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.756680    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ed4e451484feca11635366f26cff0482-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-53.ec2.internal\" (UID: \"ed4e451484feca11635366f26cff0482\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-53.ec2.internal"
Sep 17 14:04:59.812793 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:04:59.812734    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-53.ec2.internal\" not found"
Sep 17 14:04:59.909343 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.909302    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-53.ec2.internal"
Sep 17 14:04:59.913533 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:04:59.912915    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-53.ec2.internal\" not found"
Sep 17 14:04:59.913533 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:04:59.912954    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-53.ec2.internal"
Sep 17 14:05:00.013577 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:00.013539    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-53.ec2.internal\" not found"
Sep 17 14:05:00.117358 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:00.117265    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-53.ec2.internal\" not found"
Sep 17 14:05:00.221584 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:00.221536    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-53.ec2.internal\" not found"
Sep 17 14:05:00.280826 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:00.280790    2582 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 17 14:05:00.320138 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:00.320106    2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 14:05:00.320876 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:00.320862    2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 14:05:00.321838 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:00.321821    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-53.ec2.internal\" not found"
Sep 17 14:05:00.422545 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:00.422469    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-53.ec2.internal\" not found"
Sep 17 14:05:00.467724 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:00.467693    2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 14:05:00.545509 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:00.545470    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-16 13:59:59 +0000 UTC" deadline="2028-05-09 19:07:29.153396759 +0000 UTC"
Sep 17 14:05:00.545509 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:00.545501    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14405h2m28.60789914s"
Sep 17 14:05:00.554773 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:00.554746    2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-53.ec2.internal"
Sep 17 14:05:00.567867 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:05:00.567704    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded4e451484feca11635366f26cff0482.slice/crio-60214601bf38ea06c3fb9365ac14ae6686aa5303dfad5fb54724941fa5b78bb1 WatchSource:0}: Error finding container 60214601bf38ea06c3fb9365ac14ae6686aa5303dfad5fb54724941fa5b78bb1: Status 404 returned error can't find the container with id 60214601bf38ea06c3fb9365ac14ae6686aa5303dfad5fb54724941fa5b78bb1
Sep 17 14:05:00.568271 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:05:00.568253    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ed51660b0e0cea0ccab4eff146b4e99.slice/crio-fc175b6dc4533024e10c63fd65a2fbc3a971a7170cd3bf481291cc73209bf9c0 WatchSource:0}: Error finding container fc175b6dc4533024e10c63fd65a2fbc3a971a7170cd3bf481291cc73209bf9c0: Status 404 returned error can't find the container with id fc175b6dc4533024e10c63fd65a2fbc3a971a7170cd3bf481291cc73209bf9c0
Sep 17 14:05:00.573533 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:00.573431    2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 17 14:05:00.573880 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:00.573865    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 14:05:00.575303 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:00.575285    2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-53.ec2.internal"
Sep 17 14:05:00.585027 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:00.585003    2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 17 14:05:00.755864 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:00.755768    2582 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 14:05:01.327476 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.327437    2582 apiserver.go:52] "Watching apiserver"
Sep 17 14:05:01.338804 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.338772    2582 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 17 14:05:01.339181 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.339157    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-r4p4g","openshift-ovn-kubernetes/ovnkube-node-qn4cf","kube-system/global-pull-secret-syncer-7k2wk","kube-system/konnectivity-agent-2j77v","kube-system/kube-apiserver-proxy-ip-10-0-137-53.ec2.internal","openshift-cluster-node-tuning-operator/tuned-6bcj9","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-53.ec2.internal","openshift-multus/multus-t4548","openshift-network-diagnostics/network-check-target-dg4gz","openshift-network-operator/iptables-alerter-jwwq2","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gtdk","openshift-dns/node-resolver-s2nlk","openshift-image-registry/node-ca-72skg","openshift-multus/multus-additional-cni-plugins-xflkb"]
Sep 17 14:05:01.340734 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.340713    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4p4g"
Sep 17 14:05:01.340833 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:01.340809    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4p4g" podUID="059afb43-5e5e-4db9-bfa2-d874291c1c3a"
Sep 17 14:05:01.341846 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.341832    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.343023 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.343002    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7k2wk"
Sep 17 14:05:01.343119 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:01.343081    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7k2wk" podUID="31e72879-bbfc-4684-bcbb-fd79b4eec4b6"
Sep 17 14:05:01.344176 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.344159    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2j77v"
Sep 17 14:05:01.344732 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.344711    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 17 14:05:01.345263 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.345248    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 17 14:05:01.345484 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.345455    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6bcj9"
Sep 17 14:05:01.345675 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.345654    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 17 14:05:01.345764 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.345680    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 17 14:05:01.345764 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.345711    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 17 14:05:01.345872 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.345663    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-9qxwm\""
Sep 17 14:05:01.345926 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.345808    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 17 14:05:01.346590 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.346571    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-4x77w\""
Sep 17 14:05:01.346707 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.346690    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.346778 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.346732    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 17 14:05:01.346836 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.346785    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 17 14:05:01.347840 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.347823    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dg4gz"
Sep 17 14:05:01.347928 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:01.347870    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dg4gz" podUID="a2cbe70b-eb96-4779-9143-b95a1204aae2"
Sep 17 14:05:01.348043 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.348024    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 17 14:05:01.348043 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.348042    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ndpsd\""
Sep 17 14:05:01.348196 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.348088    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 17 14:05:01.349131 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.349081    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jwwq2"
Sep 17 14:05:01.349236 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.349213    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 17 14:05:01.349416 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.349395    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-5zps9\""
Sep 17 14:05:01.349504 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.349483    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 17 14:05:01.349707 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.349676    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 17 14:05:01.349831 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.349816    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 17 14:05:01.350946 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.350929    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gtdk"
Sep 17 14:05:01.351668 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.351652    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 17 14:05:01.351915 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.351895    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 17 14:05:01.352313 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.352231    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 17 14:05:01.352313 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.352267    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-s2nlk"
Sep 17 14:05:01.352587 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.352569    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7xcmx\""
Sep 17 14:05:01.353382 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.353365    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-52pb5\""
Sep 17 14:05:01.353539 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.353525    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 17 14:05:01.353734 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.353720    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 17 14:05:01.353845 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.353821    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 17 14:05:01.354040 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.354026    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-72skg"
Sep 17 14:05:01.354273 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.354259    2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 17 14:05:01.355187 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.355165    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 17 14:05:01.355590 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.355569    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 17 14:05:01.355703 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.355612    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-587tq\""
Sep 17 14:05:01.355778 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.355760    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xflkb"
Sep 17 14:05:01.356546 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.356527    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 17 14:05:01.356651 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.356582    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 17 14:05:01.357332 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.357315    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 17 14:05:01.357445 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.357423    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-bjrvl\""
Sep 17 14:05:01.358429 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.358414    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 17 14:05:01.358429 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.358422    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-wjt26\""
Sep 17 14:05:01.358764 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.358748    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 17 14:05:01.364355 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.364328    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/82aa9496-50ec-4a83-9d0c-d227df61037a-cni-binary-copy\") pod \"multus-additional-cni-plugins-xflkb\" (UID: \"82aa9496-50ec-4a83-9d0c-d227df61037a\") " pod="openshift-multus/multus-additional-cni-plugins-xflkb"
Sep 17 14:05:01.364421 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.364364    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7824d21a-4398-41f6-9303-501f38a2cfd4-host-cni-netd\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.364421 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.364392    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7824d21a-4398-41f6-9303-501f38a2cfd4-ovn-node-metrics-cert\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.364501 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.364441    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6504733b-af2d-45a5-a817-7fe57f23d503-multus-socket-dir-parent\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.364501 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.364475    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6504733b-af2d-45a5-a817-7fe57f23d503-etc-kubernetes\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.364579 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.364502    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/22d71632-956e-4285-9cc2-928ec6492277-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4gtdk\" (UID: \"22d71632-956e-4285-9cc2-928ec6492277\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gtdk"
Sep 17 14:05:01.364579 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.364525    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4462f6aa-ccc5-42d0-8623-f0dad6105560-host\") pod \"node-ca-72skg\" (UID: \"4462f6aa-ccc5-42d0-8623-f0dad6105560\") " pod="openshift-image-registry/node-ca-72skg"
Sep 17 14:05:01.364579 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.364568    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31e72879-bbfc-4684-bcbb-fd79b4eec4b6-original-pull-secret\") pod \"global-pull-secret-syncer-7k2wk\" (UID: \"31e72879-bbfc-4684-bcbb-fd79b4eec4b6\") " pod="kube-system/global-pull-secret-syncer-7k2wk"
Sep 17 14:05:01.364701 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.364596    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/22d71632-956e-4285-9cc2-928ec6492277-etc-selinux\") pod \"aws-ebs-csi-driver-node-4gtdk\" (UID: \"22d71632-956e-4285-9cc2-928ec6492277\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gtdk"
Sep 17 14:05:01.364701 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.364643    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/11334641-b1cc-4ede-aa2f-998edc2a4db0-etc-sysctl-d\") pod \"tuned-6bcj9\" (UID: \"11334641-b1cc-4ede-aa2f-998edc2a4db0\") " pod="openshift-cluster-node-tuning-operator/tuned-6bcj9"
Sep 17 14:05:01.364701 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.364675    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7824d21a-4398-41f6-9303-501f38a2cfd4-node-log\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.364808 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.364704    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thz9t\" (UniqueName: \"kubernetes.io/projected/7824d21a-4398-41f6-9303-501f38a2cfd4-kube-api-access-thz9t\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.364808 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.364730    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6504733b-af2d-45a5-a817-7fe57f23d503-host-run-multus-certs\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.364808 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.364756    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvl8x\" (UniqueName: \"kubernetes.io/projected/a2cbe70b-eb96-4779-9143-b95a1204aae2-kube-api-access-nvl8x\") pod \"network-check-target-dg4gz\" (UID: \"a2cbe70b-eb96-4779-9143-b95a1204aae2\") " pod="openshift-network-diagnostics/network-check-target-dg4gz"
Sep 17 14:05:01.364808 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.364782    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/22d71632-956e-4285-9cc2-928ec6492277-device-dir\") pod \"aws-ebs-csi-driver-node-4gtdk\" (UID: \"22d71632-956e-4285-9cc2-928ec6492277\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gtdk"
Sep 17 14:05:01.364935 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.364806    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8x42m\" (UniqueName: \"kubernetes.io/projected/4462f6aa-ccc5-42d0-8623-f0dad6105560-kube-api-access-8x42m\") pod \"node-ca-72skg\" (UID: \"4462f6aa-ccc5-42d0-8623-f0dad6105560\") " pod="openshift-image-registry/node-ca-72skg"
Sep 17 14:05:01.364935 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.364830    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2mnp\" (UniqueName: \"kubernetes.io/projected/82aa9496-50ec-4a83-9d0c-d227df61037a-kube-api-access-f2mnp\") pod \"multus-additional-cni-plugins-xflkb\" (UID: \"82aa9496-50ec-4a83-9d0c-d227df61037a\") " pod="openshift-multus/multus-additional-cni-plugins-xflkb"
Sep 17 14:05:01.364935 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.364870    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/11334641-b1cc-4ede-aa2f-998edc2a4db0-etc-sysctl-conf\") pod \"tuned-6bcj9\" (UID: \"11334641-b1cc-4ede-aa2f-998edc2a4db0\") " pod="openshift-cluster-node-tuning-operator/tuned-6bcj9"
Sep 17 14:05:01.364935 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.364894    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11334641-b1cc-4ede-aa2f-998edc2a4db0-sys\") pod \"tuned-6bcj9\" (UID: \"11334641-b1cc-4ede-aa2f-998edc2a4db0\") " pod="openshift-cluster-node-tuning-operator/tuned-6bcj9"
Sep 17 14:05:01.364935 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.364919    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1268d6c2-0f80-41e0-98d5-d72d5c03da1b-tmp-dir\") pod \"node-resolver-s2nlk\" (UID: \"1268d6c2-0f80-41e0-98d5-d72d5c03da1b\") " pod="openshift-dns/node-resolver-s2nlk"
Sep 17 14:05:01.365105 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.364943    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7824d21a-4398-41f6-9303-501f38a2cfd4-host-kubelet\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.365105 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.364969    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6504733b-af2d-45a5-a817-7fe57f23d503-host-var-lib-kubelet\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.365105 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.364993    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6504733b-af2d-45a5-a817-7fe57f23d503-multus-daemon-config\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.365105 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.365020    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4t9c\" (UniqueName: \"kubernetes.io/projected/22d71632-956e-4285-9cc2-928ec6492277-kube-api-access-t4t9c\") pod \"aws-ebs-csi-driver-node-4gtdk\" (UID: \"22d71632-956e-4285-9cc2-928ec6492277\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gtdk"
Sep 17 14:05:01.365105 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.365043    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4462f6aa-ccc5-42d0-8623-f0dad6105560-serviceca\") pod \"node-ca-72skg\" (UID: \"4462f6aa-ccc5-42d0-8623-f0dad6105560\") " pod="openshift-image-registry/node-ca-72skg"
Sep 17 14:05:01.365105 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.365073    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/82aa9496-50ec-4a83-9d0c-d227df61037a-cnibin\") pod \"multus-additional-cni-plugins-xflkb\" (UID: \"82aa9496-50ec-4a83-9d0c-d227df61037a\") " pod="openshift-multus/multus-additional-cni-plugins-xflkb"
Sep 17 14:05:01.365105 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.365098    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7824d21a-4398-41f6-9303-501f38a2cfd4-etc-openvswitch\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.365334 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.365130    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7824d21a-4398-41f6-9303-501f38a2cfd4-ovnkube-script-lib\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.365334 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.365163    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/11334641-b1cc-4ede-aa2f-998edc2a4db0-etc-sysconfig\") pod \"tuned-6bcj9\" (UID: \"11334641-b1cc-4ede-aa2f-998edc2a4db0\") " pod="openshift-cluster-node-tuning-operator/tuned-6bcj9"
Sep 17 14:05:01.365334 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.365191    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6504733b-af2d-45a5-a817-7fe57f23d503-host-var-lib-cni-bin\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.366253 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.365222    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zz6h9\" (UniqueName: \"kubernetes.io/projected/6504733b-af2d-45a5-a817-7fe57f23d503-kube-api-access-zz6h9\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.366253 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.365587    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/927c3577-20a9-4c54-8c98-0af3bcb37c06-iptables-alerter-script\") pod \"iptables-alerter-jwwq2\" (UID: \"927c3577-20a9-4c54-8c98-0af3bcb37c06\") " pod="openshift-network-operator/iptables-alerter-jwwq2"
Sep 17 14:05:01.366253 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.365626    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7824d21a-4398-41f6-9303-501f38a2cfd4-host-slash\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.366253 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.365684    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6504733b-af2d-45a5-a817-7fe57f23d503-host-run-netns\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.366253 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.365715    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f-konnectivity-ca\") pod \"konnectivity-agent-2j77v\" (UID: \"bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f\") " pod="kube-system/konnectivity-agent-2j77v"
Sep 17 14:05:01.366253 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.365738    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/11334641-b1cc-4ede-aa2f-998edc2a4db0-lib-modules\") pod \"tuned-6bcj9\" (UID: \"11334641-b1cc-4ede-aa2f-998edc2a4db0\") " pod="openshift-cluster-node-tuning-operator/tuned-6bcj9"
Sep 17 14:05:01.366253 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.365767    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6504733b-af2d-45a5-a817-7fe57f23d503-multus-conf-dir\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.366253 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.365972    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/31e72879-bbfc-4684-bcbb-fd79b4eec4b6-kubelet-config\") pod \"global-pull-secret-syncer-7k2wk\" (UID: \"31e72879-bbfc-4684-bcbb-fd79b4eec4b6\") " pod="kube-system/global-pull-secret-syncer-7k2wk"
Sep 17 14:05:01.366253 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.365984    2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 17 14:05:01.366253 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.366024    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6504733b-af2d-45a5-a817-7fe57f23d503-system-cni-dir\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.366253 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.366073    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/82aa9496-50ec-4a83-9d0c-d227df61037a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xflkb\" (UID: \"82aa9496-50ec-4a83-9d0c-d227df61037a\") " pod="openshift-multus/multus-additional-cni-plugins-xflkb"
Sep 17 14:05:01.366253 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.366109    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/059afb43-5e5e-4db9-bfa2-d874291c1c3a-metrics-certs\") pod \"network-metrics-daemon-r4p4g\" (UID: \"059afb43-5e5e-4db9-bfa2-d874291c1c3a\") " pod="openshift-multus/network-metrics-daemon-r4p4g"
Sep 17 14:05:01.366253 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.366140    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7824d21a-4398-41f6-9303-501f38a2cfd4-run-openvswitch\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.366253 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.366170    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7824d21a-4398-41f6-9303-501f38a2cfd4-host-run-ovn-kubernetes\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.366253 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.366195    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7824d21a-4398-41f6-9303-501f38a2cfd4-host-cni-bin\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.366999 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.366337    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7824d21a-4398-41f6-9303-501f38a2cfd4-env-overrides\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.366999 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.366377    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/82aa9496-50ec-4a83-9d0c-d227df61037a-system-cni-dir\") pod \"multus-additional-cni-plugins-xflkb\" (UID: \"82aa9496-50ec-4a83-9d0c-d227df61037a\") " pod="openshift-multus/multus-additional-cni-plugins-xflkb"
Sep 17 14:05:01.366999 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.366403    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/82aa9496-50ec-4a83-9d0c-d227df61037a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xflkb\" (UID: \"82aa9496-50ec-4a83-9d0c-d227df61037a\") " pod="openshift-multus/multus-additional-cni-plugins-xflkb"
Sep 17 14:05:01.366999 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.366437    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7824d21a-4398-41f6-9303-501f38a2cfd4-systemd-units\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.366999 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.366484    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f-agent-certs\") pod \"konnectivity-agent-2j77v\" (UID: \"bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f\") " pod="kube-system/konnectivity-agent-2j77v"
Sep 17 14:05:01.366999 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.366515    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11334641-b1cc-4ede-aa2f-998edc2a4db0-tmp\") pod \"tuned-6bcj9\" (UID: \"11334641-b1cc-4ede-aa2f-998edc2a4db0\") " pod="openshift-cluster-node-tuning-operator/tuned-6bcj9"
Sep 17 14:05:01.366999 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.366546    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6504733b-af2d-45a5-a817-7fe57f23d503-host-run-k8s-cni-cncf-io\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.366999 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.366572    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/22d71632-956e-4285-9cc2-928ec6492277-registration-dir\") pod \"aws-ebs-csi-driver-node-4gtdk\" (UID: \"22d71632-956e-4285-9cc2-928ec6492277\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gtdk"
Sep 17 14:05:01.366999 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.366601    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/11334641-b1cc-4ede-aa2f-998edc2a4db0-run\") pod \"tuned-6bcj9\" (UID: \"11334641-b1cc-4ede-aa2f-998edc2a4db0\") " pod="openshift-cluster-node-tuning-operator/tuned-6bcj9"
Sep 17 14:05:01.366999 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.366647    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpbsl\" (UniqueName: \"kubernetes.io/projected/11334641-b1cc-4ede-aa2f-998edc2a4db0-kube-api-access-hpbsl\") pod \"tuned-6bcj9\" (UID: \"11334641-b1cc-4ede-aa2f-998edc2a4db0\") " pod="openshift-cluster-node-tuning-operator/tuned-6bcj9"
Sep 17 14:05:01.366999 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.366677    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6504733b-af2d-45a5-a817-7fe57f23d503-multus-cni-dir\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.366999 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.366708    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7824d21a-4398-41f6-9303-501f38a2cfd4-run-ovn\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.366999 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.366739    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7824d21a-4398-41f6-9303-501f38a2cfd4-log-socket\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.366999 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.366764    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7824d21a-4398-41f6-9303-501f38a2cfd4-ovnkube-config\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.366999 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.366794    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/11334641-b1cc-4ede-aa2f-998edc2a4db0-etc-modprobe-d\") pod \"tuned-6bcj9\" (UID: \"11334641-b1cc-4ede-aa2f-998edc2a4db0\") " pod="openshift-cluster-node-tuning-operator/tuned-6bcj9"
Sep 17 14:05:01.366999 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.366840    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11334641-b1cc-4ede-aa2f-998edc2a4db0-etc-kubernetes\") pod \"tuned-6bcj9\" (UID: \"11334641-b1cc-4ede-aa2f-998edc2a4db0\") " pod="openshift-cluster-node-tuning-operator/tuned-6bcj9"
Sep 17 14:05:01.367849 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.366868    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6504733b-af2d-45a5-a817-7fe57f23d503-cnibin\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.367849 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.366897    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6504733b-af2d-45a5-a817-7fe57f23d503-host-var-lib-cni-multus\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.367849 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.366933    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6504733b-af2d-45a5-a817-7fe57f23d503-hostroot\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.367849 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.366979    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/31e72879-bbfc-4684-bcbb-fd79b4eec4b6-dbus\") pod \"global-pull-secret-syncer-7k2wk\" (UID: \"31e72879-bbfc-4684-bcbb-fd79b4eec4b6\") " pod="kube-system/global-pull-secret-syncer-7k2wk"
Sep 17 14:05:01.367849 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.367013    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/927c3577-20a9-4c54-8c98-0af3bcb37c06-host-slash\") pod \"iptables-alerter-jwwq2\" (UID: \"927c3577-20a9-4c54-8c98-0af3bcb37c06\") " pod="openshift-network-operator/iptables-alerter-jwwq2"
Sep 17 14:05:01.367849 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.367043    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghlb2\" (UniqueName: \"kubernetes.io/projected/927c3577-20a9-4c54-8c98-0af3bcb37c06-kube-api-access-ghlb2\") pod \"iptables-alerter-jwwq2\" (UID: \"927c3577-20a9-4c54-8c98-0af3bcb37c06\") " pod="openshift-network-operator/iptables-alerter-jwwq2"
Sep 17 14:05:01.367849 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.367076    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/82aa9496-50ec-4a83-9d0c-d227df61037a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xflkb\" (UID: \"82aa9496-50ec-4a83-9d0c-d227df61037a\") " pod="openshift-multus/multus-additional-cni-plugins-xflkb"
Sep 17 14:05:01.367849 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.367103    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/11334641-b1cc-4ede-aa2f-998edc2a4db0-etc-systemd\") pod \"tuned-6bcj9\" (UID: \"11334641-b1cc-4ede-aa2f-998edc2a4db0\") " pod="openshift-cluster-node-tuning-operator/tuned-6bcj9"
Sep 17 14:05:01.367849 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.367131    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11334641-b1cc-4ede-aa2f-998edc2a4db0-var-lib-kubelet\") pod \"tuned-6bcj9\" (UID: \"11334641-b1cc-4ede-aa2f-998edc2a4db0\") " pod="openshift-cluster-node-tuning-operator/tuned-6bcj9"
Sep 17 14:05:01.367849 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.367160    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/11334641-b1cc-4ede-aa2f-998edc2a4db0-host\") pod \"tuned-6bcj9\" (UID: \"11334641-b1cc-4ede-aa2f-998edc2a4db0\") " pod="openshift-cluster-node-tuning-operator/tuned-6bcj9"
Sep 17 14:05:01.367849 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.367187    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/22d71632-956e-4285-9cc2-928ec6492277-socket-dir\") pod \"aws-ebs-csi-driver-node-4gtdk\" (UID: \"22d71632-956e-4285-9cc2-928ec6492277\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gtdk"
Sep 17 14:05:01.367849 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.367213    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/22d71632-956e-4285-9cc2-928ec6492277-sys-fs\") pod \"aws-ebs-csi-driver-node-4gtdk\" (UID: \"22d71632-956e-4285-9cc2-928ec6492277\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gtdk"
Sep 17 14:05:01.367849 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.367242    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7824d21a-4398-41f6-9303-501f38a2cfd4-host-run-netns\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.367849 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.367274    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7824d21a-4398-41f6-9303-501f38a2cfd4-run-systemd\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.367849 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.367301    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/11334641-b1cc-4ede-aa2f-998edc2a4db0-etc-tuned\") pod \"tuned-6bcj9\" (UID: \"11334641-b1cc-4ede-aa2f-998edc2a4db0\") " pod="openshift-cluster-node-tuning-operator/tuned-6bcj9"
Sep 17 14:05:01.367849 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.367331    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1268d6c2-0f80-41e0-98d5-d72d5c03da1b-hosts-file\") pod \"node-resolver-s2nlk\" (UID: \"1268d6c2-0f80-41e0-98d5-d72d5c03da1b\") " pod="openshift-dns/node-resolver-s2nlk"
Sep 17 14:05:01.367849 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.367363    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4czcb\" (UniqueName: \"kubernetes.io/projected/1268d6c2-0f80-41e0-98d5-d72d5c03da1b-kube-api-access-4czcb\") pod \"node-resolver-s2nlk\" (UID: \"1268d6c2-0f80-41e0-98d5-d72d5c03da1b\") " pod="openshift-dns/node-resolver-s2nlk"
Sep 17 14:05:01.368613 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.367392    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6504733b-af2d-45a5-a817-7fe57f23d503-cni-binary-copy\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.368613 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.367418    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66f8r\" (UniqueName: \"kubernetes.io/projected/059afb43-5e5e-4db9-bfa2-d874291c1c3a-kube-api-access-66f8r\") pod \"network-metrics-daemon-r4p4g\" (UID: \"059afb43-5e5e-4db9-bfa2-d874291c1c3a\") " pod="openshift-multus/network-metrics-daemon-r4p4g"
Sep 17 14:05:01.368613 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.367453    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7824d21a-4398-41f6-9303-501f38a2cfd4-var-lib-openvswitch\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.368613 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.367484    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7824d21a-4398-41f6-9303-501f38a2cfd4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.454366 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.454334    2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-mrtt8"
Sep 17 14:05:01.456355 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.456333    2582 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 17 14:05:01.462793 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.462771    2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-mrtt8"
Sep 17 14:05:01.467687 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.467657    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1268d6c2-0f80-41e0-98d5-d72d5c03da1b-tmp-dir\") pod \"node-resolver-s2nlk\" (UID: \"1268d6c2-0f80-41e0-98d5-d72d5c03da1b\") " pod="openshift-dns/node-resolver-s2nlk"
Sep 17 14:05:01.467799 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.467755    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7824d21a-4398-41f6-9303-501f38a2cfd4-host-kubelet\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.467799 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.467786    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6504733b-af2d-45a5-a817-7fe57f23d503-host-var-lib-kubelet\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.467919 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.467810    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6504733b-af2d-45a5-a817-7fe57f23d503-multus-daemon-config\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.467919 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.467836    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4t9c\" (UniqueName: \"kubernetes.io/projected/22d71632-956e-4285-9cc2-928ec6492277-kube-api-access-t4t9c\") pod \"aws-ebs-csi-driver-node-4gtdk\" (UID: \"22d71632-956e-4285-9cc2-928ec6492277\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gtdk"
Sep 17 14:05:01.467919 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.467860    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4462f6aa-ccc5-42d0-8623-f0dad6105560-serviceca\") pod \"node-ca-72skg\" (UID: \"4462f6aa-ccc5-42d0-8623-f0dad6105560\") " pod="openshift-image-registry/node-ca-72skg"
Sep 17 14:05:01.467919 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.467887    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/82aa9496-50ec-4a83-9d0c-d227df61037a-cnibin\") pod \"multus-additional-cni-plugins-xflkb\" (UID: \"82aa9496-50ec-4a83-9d0c-d227df61037a\") " pod="openshift-multus/multus-additional-cni-plugins-xflkb"
Sep 17 14:05:01.467919 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.467899    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6504733b-af2d-45a5-a817-7fe57f23d503-host-var-lib-kubelet\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.467919 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.467910    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7824d21a-4398-41f6-9303-501f38a2cfd4-etc-openvswitch\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.467919 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.467861    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7824d21a-4398-41f6-9303-501f38a2cfd4-host-kubelet\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.468203 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.467955    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7824d21a-4398-41f6-9303-501f38a2cfd4-ovnkube-script-lib\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.468203 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.468011    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/11334641-b1cc-4ede-aa2f-998edc2a4db0-etc-sysconfig\") pod \"tuned-6bcj9\" (UID: \"11334641-b1cc-4ede-aa2f-998edc2a4db0\") " pod="openshift-cluster-node-tuning-operator/tuned-6bcj9"
Sep 17 14:05:01.468203 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.468037    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6504733b-af2d-45a5-a817-7fe57f23d503-host-var-lib-cni-bin\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.468203 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.468076    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zz6h9\" (UniqueName: \"kubernetes.io/projected/6504733b-af2d-45a5-a817-7fe57f23d503-kube-api-access-zz6h9\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.468203 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.468098    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1268d6c2-0f80-41e0-98d5-d72d5c03da1b-tmp-dir\") pod \"node-resolver-s2nlk\" (UID: \"1268d6c2-0f80-41e0-98d5-d72d5c03da1b\") " pod="openshift-dns/node-resolver-s2nlk"
Sep 17 14:05:01.468203 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.468102    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/927c3577-20a9-4c54-8c98-0af3bcb37c06-iptables-alerter-script\") pod \"iptables-alerter-jwwq2\" (UID: \"927c3577-20a9-4c54-8c98-0af3bcb37c06\") " pod="openshift-network-operator/iptables-alerter-jwwq2"
Sep 17 14:05:01.468203 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.468163    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7824d21a-4398-41f6-9303-501f38a2cfd4-host-slash\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.468203 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.468191    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6504733b-af2d-45a5-a817-7fe57f23d503-host-run-netns\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.468544 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.468217    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f-konnectivity-ca\") pod \"konnectivity-agent-2j77v\" (UID: \"bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f\") " pod="kube-system/konnectivity-agent-2j77v"
Sep 17 14:05:01.468544 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.468242    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/11334641-b1cc-4ede-aa2f-998edc2a4db0-lib-modules\") pod \"tuned-6bcj9\" (UID: \"11334641-b1cc-4ede-aa2f-998edc2a4db0\") " pod="openshift-cluster-node-tuning-operator/tuned-6bcj9"
Sep 17 14:05:01.468544 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.468269    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6504733b-af2d-45a5-a817-7fe57f23d503-multus-conf-dir\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.468544 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.468300    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/31e72879-bbfc-4684-bcbb-fd79b4eec4b6-kubelet-config\") pod \"global-pull-secret-syncer-7k2wk\" (UID: \"31e72879-bbfc-4684-bcbb-fd79b4eec4b6\") " pod="kube-system/global-pull-secret-syncer-7k2wk"
Sep 17 14:05:01.468544 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.468325    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6504733b-af2d-45a5-a817-7fe57f23d503-system-cni-dir\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.468544 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.468353    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/82aa9496-50ec-4a83-9d0c-d227df61037a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xflkb\" (UID: \"82aa9496-50ec-4a83-9d0c-d227df61037a\") " pod="openshift-multus/multus-additional-cni-plugins-xflkb"
Sep 17 14:05:01.468544 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.468383    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/059afb43-5e5e-4db9-bfa2-d874291c1c3a-metrics-certs\") pod \"network-metrics-daemon-r4p4g\" (UID: \"059afb43-5e5e-4db9-bfa2-d874291c1c3a\") " pod="openshift-multus/network-metrics-daemon-r4p4g"
Sep 17 14:05:01.468544 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.468409    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7824d21a-4398-41f6-9303-501f38a2cfd4-run-openvswitch\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.468544 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.468435    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7824d21a-4398-41f6-9303-501f38a2cfd4-host-run-ovn-kubernetes\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.468544 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.468460    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7824d21a-4398-41f6-9303-501f38a2cfd4-host-cni-bin\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.468544 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.468485    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7824d21a-4398-41f6-9303-501f38a2cfd4-env-overrides\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.468544 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.468513    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/82aa9496-50ec-4a83-9d0c-d227df61037a-system-cni-dir\") pod \"multus-additional-cni-plugins-xflkb\" (UID: \"82aa9496-50ec-4a83-9d0c-d227df61037a\") " pod="openshift-multus/multus-additional-cni-plugins-xflkb"
Sep 17 14:05:01.468544 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.468541    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/82aa9496-50ec-4a83-9d0c-d227df61037a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xflkb\" (UID: \"82aa9496-50ec-4a83-9d0c-d227df61037a\") " pod="openshift-multus/multus-additional-cni-plugins-xflkb"
Sep 17 14:05:01.469239 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.468569    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7824d21a-4398-41f6-9303-501f38a2cfd4-systemd-units\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.469239 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.468597    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f-agent-certs\") pod \"konnectivity-agent-2j77v\" (UID: \"bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f\") " pod="kube-system/konnectivity-agent-2j77v"
Sep 17 14:05:01.469239 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.468621    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11334641-b1cc-4ede-aa2f-998edc2a4db0-tmp\") pod \"tuned-6bcj9\" (UID: \"11334641-b1cc-4ede-aa2f-998edc2a4db0\") " pod="openshift-cluster-node-tuning-operator/tuned-6bcj9"
Sep 17 14:05:01.469239 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.468647    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6504733b-af2d-45a5-a817-7fe57f23d503-host-run-k8s-cni-cncf-io\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.469239 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.468685    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4462f6aa-ccc5-42d0-8623-f0dad6105560-serviceca\") pod \"node-ca-72skg\" (UID: \"4462f6aa-ccc5-42d0-8623-f0dad6105560\") " pod="openshift-image-registry/node-ca-72skg"
Sep 17 14:05:01.469239 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.468693    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/22d71632-956e-4285-9cc2-928ec6492277-registration-dir\") pod \"aws-ebs-csi-driver-node-4gtdk\" (UID: \"22d71632-956e-4285-9cc2-928ec6492277\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gtdk"
Sep 17 14:05:01.469239 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.468681    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/927c3577-20a9-4c54-8c98-0af3bcb37c06-iptables-alerter-script\") pod \"iptables-alerter-jwwq2\" (UID: \"927c3577-20a9-4c54-8c98-0af3bcb37c06\") " pod="openshift-network-operator/iptables-alerter-jwwq2"
Sep 17 14:05:01.469239 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.468719    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/11334641-b1cc-4ede-aa2f-998edc2a4db0-run\") pod \"tuned-6bcj9\" (UID: \"11334641-b1cc-4ede-aa2f-998edc2a4db0\") " pod="openshift-cluster-node-tuning-operator/tuned-6bcj9"
Sep 17 14:05:01.469239 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.468746    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpbsl\" (UniqueName: \"kubernetes.io/projected/11334641-b1cc-4ede-aa2f-998edc2a4db0-kube-api-access-hpbsl\") pod \"tuned-6bcj9\" (UID: \"11334641-b1cc-4ede-aa2f-998edc2a4db0\") " pod="openshift-cluster-node-tuning-operator/tuned-6bcj9"
Sep 17 14:05:01.469239 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.468772    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6504733b-af2d-45a5-a817-7fe57f23d503-multus-cni-dir\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.469239 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.468783    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7824d21a-4398-41f6-9303-501f38a2cfd4-run-openvswitch\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.469239 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.468799    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7824d21a-4398-41f6-9303-501f38a2cfd4-run-ovn\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.469239 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.468829    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7824d21a-4398-41f6-9303-501f38a2cfd4-host-slash\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.469239 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.468841    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7824d21a-4398-41f6-9303-501f38a2cfd4-log-socket\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.469239 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.468868    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7824d21a-4398-41f6-9303-501f38a2cfd4-ovnkube-config\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.469239 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.468874    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6504733b-af2d-45a5-a817-7fe57f23d503-host-run-netns\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.469239 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.468895    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/11334641-b1cc-4ede-aa2f-998edc2a4db0-etc-modprobe-d\") pod \"tuned-6bcj9\" (UID: \"11334641-b1cc-4ede-aa2f-998edc2a4db0\") " pod="openshift-cluster-node-tuning-operator/tuned-6bcj9"
Sep 17 14:05:01.469239 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.468923    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11334641-b1cc-4ede-aa2f-998edc2a4db0-etc-kubernetes\") pod \"tuned-6bcj9\" (UID: \"11334641-b1cc-4ede-aa2f-998edc2a4db0\") " pod="openshift-cluster-node-tuning-operator/tuned-6bcj9"
Sep 17 14:05:01.470180 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.468947    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6504733b-af2d-45a5-a817-7fe57f23d503-cnibin\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.470180 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.468976    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6504733b-af2d-45a5-a817-7fe57f23d503-host-var-lib-cni-multus\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.470180 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.469002    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6504733b-af2d-45a5-a817-7fe57f23d503-hostroot\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.470180 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.469027    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/31e72879-bbfc-4684-bcbb-fd79b4eec4b6-dbus\") pod \"global-pull-secret-syncer-7k2wk\" (UID: \"31e72879-bbfc-4684-bcbb-fd79b4eec4b6\") " pod="kube-system/global-pull-secret-syncer-7k2wk"
Sep 17 14:05:01.470180 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.469057    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/927c3577-20a9-4c54-8c98-0af3bcb37c06-host-slash\") pod \"iptables-alerter-jwwq2\" (UID: \"927c3577-20a9-4c54-8c98-0af3bcb37c06\") " pod="openshift-network-operator/iptables-alerter-jwwq2"
Sep 17 14:05:01.470180 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.469083    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghlb2\" (UniqueName: \"kubernetes.io/projected/927c3577-20a9-4c54-8c98-0af3bcb37c06-kube-api-access-ghlb2\") pod \"iptables-alerter-jwwq2\" (UID: \"927c3577-20a9-4c54-8c98-0af3bcb37c06\") " pod="openshift-network-operator/iptables-alerter-jwwq2"
Sep 17 14:05:01.470180 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.469113    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/82aa9496-50ec-4a83-9d0c-d227df61037a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xflkb\" (UID: \"82aa9496-50ec-4a83-9d0c-d227df61037a\") " pod="openshift-multus/multus-additional-cni-plugins-xflkb"
Sep 17 14:05:01.470180 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.469139    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/11334641-b1cc-4ede-aa2f-998edc2a4db0-etc-systemd\") pod \"tuned-6bcj9\" (UID: \"11334641-b1cc-4ede-aa2f-998edc2a4db0\") " pod="openshift-cluster-node-tuning-operator/tuned-6bcj9"
Sep 17 14:05:01.470180 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.469184    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11334641-b1cc-4ede-aa2f-998edc2a4db0-var-lib-kubelet\") pod \"tuned-6bcj9\" (UID: \"11334641-b1cc-4ede-aa2f-998edc2a4db0\") " pod="openshift-cluster-node-tuning-operator/tuned-6bcj9"
Sep 17 14:05:01.470180 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.469209    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/11334641-b1cc-4ede-aa2f-998edc2a4db0-host\") pod \"tuned-6bcj9\" (UID: \"11334641-b1cc-4ede-aa2f-998edc2a4db0\") " pod="openshift-cluster-node-tuning-operator/tuned-6bcj9"
Sep 17 14:05:01.470180 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.469229    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6504733b-af2d-45a5-a817-7fe57f23d503-multus-daemon-config\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.470180 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.469237    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/22d71632-956e-4285-9cc2-928ec6492277-socket-dir\") pod \"aws-ebs-csi-driver-node-4gtdk\" (UID: \"22d71632-956e-4285-9cc2-928ec6492277\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gtdk"
Sep 17 14:05:01.470180 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.469263    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/22d71632-956e-4285-9cc2-928ec6492277-sys-fs\") pod \"aws-ebs-csi-driver-node-4gtdk\" (UID: \"22d71632-956e-4285-9cc2-928ec6492277\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gtdk"
Sep 17 14:05:01.470180 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.469286    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/82aa9496-50ec-4a83-9d0c-d227df61037a-cnibin\") pod \"multus-additional-cni-plugins-xflkb\" (UID: \"82aa9496-50ec-4a83-9d0c-d227df61037a\") " pod="openshift-multus/multus-additional-cni-plugins-xflkb"
Sep 17 14:05:01.470180 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.469290    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7824d21a-4398-41f6-9303-501f38a2cfd4-host-run-netns\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.470180 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.469330    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7824d21a-4398-41f6-9303-501f38a2cfd4-run-systemd\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.470180 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.469330    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7824d21a-4398-41f6-9303-501f38a2cfd4-etc-openvswitch\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.471052 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.469389    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7824d21a-4398-41f6-9303-501f38a2cfd4-host-run-ovn-kubernetes\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.471052 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.469413    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f-konnectivity-ca\") pod \"konnectivity-agent-2j77v\" (UID: \"bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f\") " pod="kube-system/konnectivity-agent-2j77v"
Sep 17 14:05:01.471052 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.469431    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7824d21a-4398-41f6-9303-501f38a2cfd4-host-cni-bin\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.471052 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.469480    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/82aa9496-50ec-4a83-9d0c-d227df61037a-system-cni-dir\") pod \"multus-additional-cni-plugins-xflkb\" (UID: \"82aa9496-50ec-4a83-9d0c-d227df61037a\") " pod="openshift-multus/multus-additional-cni-plugins-xflkb"
Sep 17 14:05:01.471052 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.469636    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/11334641-b1cc-4ede-aa2f-998edc2a4db0-lib-modules\") pod \"tuned-6bcj9\" (UID: \"11334641-b1cc-4ede-aa2f-998edc2a4db0\") " pod="openshift-cluster-node-tuning-operator/tuned-6bcj9"
Sep 17 14:05:01.471052 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.470019    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/82aa9496-50ec-4a83-9d0c-d227df61037a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xflkb\" (UID: \"82aa9496-50ec-4a83-9d0c-d227df61037a\") " pod="openshift-multus/multus-additional-cni-plugins-xflkb"
Sep 17 14:05:01.471052 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.470060    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7824d21a-4398-41f6-9303-501f38a2cfd4-env-overrides\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.471052 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.470085    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7824d21a-4398-41f6-9303-501f38a2cfd4-systemd-units\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.471052 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.470140    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/11334641-b1cc-4ede-aa2f-998edc2a4db0-etc-sysconfig\") pod \"tuned-6bcj9\" (UID: \"11334641-b1cc-4ede-aa2f-998edc2a4db0\") " pod="openshift-cluster-node-tuning-operator/tuned-6bcj9"
Sep 17 14:05:01.471052 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.470146    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/31e72879-bbfc-4684-bcbb-fd79b4eec4b6-kubelet-config\") pod \"global-pull-secret-syncer-7k2wk\" (UID: \"31e72879-bbfc-4684-bcbb-fd79b4eec4b6\") " pod="kube-system/global-pull-secret-syncer-7k2wk"
Sep 17 14:05:01.471052 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.470388    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6504733b-af2d-45a5-a817-7fe57f23d503-host-var-lib-cni-bin\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.471052 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.470426    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6504733b-af2d-45a5-a817-7fe57f23d503-multus-conf-dir\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.471052 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:01.470503    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 14:05:01.471052 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:01.470589    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/059afb43-5e5e-4db9-bfa2-d874291c1c3a-metrics-certs podName:059afb43-5e5e-4db9-bfa2-d874291c1c3a nodeName:}" failed. No retries permitted until 2026-09-17 14:05:01.970554461 +0000 UTC m=+2.972097939 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/059afb43-5e5e-4db9-bfa2-d874291c1c3a-metrics-certs") pod "network-metrics-daemon-r4p4g" (UID: "059afb43-5e5e-4db9-bfa2-d874291c1c3a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 14:05:01.471052 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.470593    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/82aa9496-50ec-4a83-9d0c-d227df61037a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xflkb\" (UID: \"82aa9496-50ec-4a83-9d0c-d227df61037a\") " pod="openshift-multus/multus-additional-cni-plugins-xflkb"
Sep 17 14:05:01.471052 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.470664    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6504733b-af2d-45a5-a817-7fe57f23d503-hostroot\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.471052 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.470762    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7824d21a-4398-41f6-9303-501f38a2cfd4-run-ovn\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.471989 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.470762    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7824d21a-4398-41f6-9303-501f38a2cfd4-log-socket\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.471989 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.470827    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6504733b-af2d-45a5-a817-7fe57f23d503-host-run-k8s-cni-cncf-io\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.471989 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.470819    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6504733b-af2d-45a5-a817-7fe57f23d503-cnibin\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.471989 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.470857    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11334641-b1cc-4ede-aa2f-998edc2a4db0-etc-kubernetes\") pod \"tuned-6bcj9\" (UID: \"11334641-b1cc-4ede-aa2f-998edc2a4db0\") " pod="openshift-cluster-node-tuning-operator/tuned-6bcj9"
Sep 17 14:05:01.471989 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.470858    2582 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 17 14:05:01.471989 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.470891    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6504733b-af2d-45a5-a817-7fe57f23d503-host-var-lib-cni-multus\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.471989 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.470964    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/11334641-b1cc-4ede-aa2f-998edc2a4db0-run\") pod \"tuned-6bcj9\" (UID: \"11334641-b1cc-4ede-aa2f-998edc2a4db0\") " pod="openshift-cluster-node-tuning-operator/tuned-6bcj9"
Sep 17 14:05:01.471989 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.470975    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6504733b-af2d-45a5-a817-7fe57f23d503-multus-cni-dir\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.471989 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.471013    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/22d71632-956e-4285-9cc2-928ec6492277-registration-dir\") pod \"aws-ebs-csi-driver-node-4gtdk\" (UID: \"22d71632-956e-4285-9cc2-928ec6492277\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gtdk"
Sep 17 14:05:01.471989 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.471248    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/11334641-b1cc-4ede-aa2f-998edc2a4db0-host\") pod \"tuned-6bcj9\" (UID: \"11334641-b1cc-4ede-aa2f-998edc2a4db0\") " pod="openshift-cluster-node-tuning-operator/tuned-6bcj9"
Sep 17 14:05:01.471989 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.471318    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/31e72879-bbfc-4684-bcbb-fd79b4eec4b6-dbus\") pod \"global-pull-secret-syncer-7k2wk\" (UID: \"31e72879-bbfc-4684-bcbb-fd79b4eec4b6\") " pod="kube-system/global-pull-secret-syncer-7k2wk"
Sep 17 14:05:01.471989 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.471324    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/11334641-b1cc-4ede-aa2f-998edc2a4db0-etc-modprobe-d\") pod \"tuned-6bcj9\" (UID: \"11334641-b1cc-4ede-aa2f-998edc2a4db0\") " pod="openshift-cluster-node-tuning-operator/tuned-6bcj9"
Sep 17 14:05:01.471989 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.471371    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11334641-b1cc-4ede-aa2f-998edc2a4db0-var-lib-kubelet\") pod \"tuned-6bcj9\" (UID: \"11334641-b1cc-4ede-aa2f-998edc2a4db0\") " pod="openshift-cluster-node-tuning-operator/tuned-6bcj9"
Sep 17 14:05:01.471989 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.470088    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7824d21a-4398-41f6-9303-501f38a2cfd4-ovnkube-script-lib\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.471989 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.471391    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7824d21a-4398-41f6-9303-501f38a2cfd4-host-run-netns\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.471989 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.471396    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7824d21a-4398-41f6-9303-501f38a2cfd4-ovnkube-config\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.471989 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.471410    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/11334641-b1cc-4ede-aa2f-998edc2a4db0-etc-systemd\") pod \"tuned-6bcj9\" (UID: \"11334641-b1cc-4ede-aa2f-998edc2a4db0\") " pod="openshift-cluster-node-tuning-operator/tuned-6bcj9"
Sep 17 14:05:01.471989 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.471422    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/11334641-b1cc-4ede-aa2f-998edc2a4db0-etc-tuned\") pod \"tuned-6bcj9\" (UID: \"11334641-b1cc-4ede-aa2f-998edc2a4db0\") " pod="openshift-cluster-node-tuning-operator/tuned-6bcj9"
Sep 17 14:05:01.472964 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.471433    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7824d21a-4398-41f6-9303-501f38a2cfd4-run-systemd\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.472964 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.471454    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/82aa9496-50ec-4a83-9d0c-d227df61037a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xflkb\" (UID: \"82aa9496-50ec-4a83-9d0c-d227df61037a\") " pod="openshift-multus/multus-additional-cni-plugins-xflkb"
Sep 17 14:05:01.472964 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.471464    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1268d6c2-0f80-41e0-98d5-d72d5c03da1b-hosts-file\") pod \"node-resolver-s2nlk\" (UID: \"1268d6c2-0f80-41e0-98d5-d72d5c03da1b\") " pod="openshift-dns/node-resolver-s2nlk"
Sep 17 14:05:01.472964 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.471495    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/22d71632-956e-4285-9cc2-928ec6492277-sys-fs\") pod \"aws-ebs-csi-driver-node-4gtdk\" (UID: \"22d71632-956e-4285-9cc2-928ec6492277\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gtdk"
Sep 17 14:05:01.472964 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.471498    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4czcb\" (UniqueName: \"kubernetes.io/projected/1268d6c2-0f80-41e0-98d5-d72d5c03da1b-kube-api-access-4czcb\") pod \"node-resolver-s2nlk\" (UID: \"1268d6c2-0f80-41e0-98d5-d72d5c03da1b\") " pod="openshift-dns/node-resolver-s2nlk"
Sep 17 14:05:01.472964 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.471509    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/22d71632-956e-4285-9cc2-928ec6492277-socket-dir\") pod \"aws-ebs-csi-driver-node-4gtdk\" (UID: \"22d71632-956e-4285-9cc2-928ec6492277\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gtdk"
Sep 17 14:05:01.472964 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.471527    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6504733b-af2d-45a5-a817-7fe57f23d503-cni-binary-copy\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.472964 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.471543    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1268d6c2-0f80-41e0-98d5-d72d5c03da1b-hosts-file\") pod \"node-resolver-s2nlk\" (UID: \"1268d6c2-0f80-41e0-98d5-d72d5c03da1b\") " pod="openshift-dns/node-resolver-s2nlk"
Sep 17 14:05:01.472964 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.471555    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66f8r\" (UniqueName: \"kubernetes.io/projected/059afb43-5e5e-4db9-bfa2-d874291c1c3a-kube-api-access-66f8r\") pod \"network-metrics-daemon-r4p4g\" (UID: \"059afb43-5e5e-4db9-bfa2-d874291c1c3a\") " pod="openshift-multus/network-metrics-daemon-r4p4g"
Sep 17 14:05:01.472964 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.471563    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/927c3577-20a9-4c54-8c98-0af3bcb37c06-host-slash\") pod \"iptables-alerter-jwwq2\" (UID: \"927c3577-20a9-4c54-8c98-0af3bcb37c06\") " pod="openshift-network-operator/iptables-alerter-jwwq2"
Sep 17 14:05:01.472964 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.471582    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7824d21a-4398-41f6-9303-501f38a2cfd4-var-lib-openvswitch\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.472964 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.471611    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7824d21a-4398-41f6-9303-501f38a2cfd4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.472964 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.471638    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/82aa9496-50ec-4a83-9d0c-d227df61037a-cni-binary-copy\") pod \"multus-additional-cni-plugins-xflkb\" (UID: \"82aa9496-50ec-4a83-9d0c-d227df61037a\") " pod="openshift-multus/multus-additional-cni-plugins-xflkb"
Sep 17 14:05:01.472964 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.471663    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7824d21a-4398-41f6-9303-501f38a2cfd4-host-cni-netd\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.472964 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.471716    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7824d21a-4398-41f6-9303-501f38a2cfd4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.472964 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.471741    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7824d21a-4398-41f6-9303-501f38a2cfd4-ovn-node-metrics-cert\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.472964 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.471776    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6504733b-af2d-45a5-a817-7fe57f23d503-multus-socket-dir-parent\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.473892 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.471791    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7824d21a-4398-41f6-9303-501f38a2cfd4-var-lib-openvswitch\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.473892 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.471804    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6504733b-af2d-45a5-a817-7fe57f23d503-etc-kubernetes\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.473892 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.471746    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6504733b-af2d-45a5-a817-7fe57f23d503-system-cni-dir\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.473892 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.471860    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6504733b-af2d-45a5-a817-7fe57f23d503-etc-kubernetes\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.473892 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.471920    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6504733b-af2d-45a5-a817-7fe57f23d503-multus-socket-dir-parent\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.473892 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.471946    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/22d71632-956e-4285-9cc2-928ec6492277-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4gtdk\" (UID: \"22d71632-956e-4285-9cc2-928ec6492277\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gtdk"
Sep 17 14:05:01.473892 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.471972    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4462f6aa-ccc5-42d0-8623-f0dad6105560-host\") pod \"node-ca-72skg\" (UID: \"4462f6aa-ccc5-42d0-8623-f0dad6105560\") " pod="openshift-image-registry/node-ca-72skg"
Sep 17 14:05:01.473892 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.472000    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31e72879-bbfc-4684-bcbb-fd79b4eec4b6-original-pull-secret\") pod \"global-pull-secret-syncer-7k2wk\" (UID: \"31e72879-bbfc-4684-bcbb-fd79b4eec4b6\") " pod="kube-system/global-pull-secret-syncer-7k2wk"
Sep 17 14:05:01.473892 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.472028    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/22d71632-956e-4285-9cc2-928ec6492277-etc-selinux\") pod \"aws-ebs-csi-driver-node-4gtdk\" (UID: \"22d71632-956e-4285-9cc2-928ec6492277\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gtdk"
Sep 17 14:05:01.473892 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.472055    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/11334641-b1cc-4ede-aa2f-998edc2a4db0-etc-sysctl-d\") pod \"tuned-6bcj9\" (UID: \"11334641-b1cc-4ede-aa2f-998edc2a4db0\") " pod="openshift-cluster-node-tuning-operator/tuned-6bcj9"
Sep 17 14:05:01.473892 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.472064    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4462f6aa-ccc5-42d0-8623-f0dad6105560-host\") pod \"node-ca-72skg\" (UID: \"4462f6aa-ccc5-42d0-8623-f0dad6105560\") " pod="openshift-image-registry/node-ca-72skg"
Sep 17 14:05:01.473892 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.472085    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7824d21a-4398-41f6-9303-501f38a2cfd4-node-log\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.473892 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.472115    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thz9t\" (UniqueName: \"kubernetes.io/projected/7824d21a-4398-41f6-9303-501f38a2cfd4-kube-api-access-thz9t\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.473892 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.472135    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/22d71632-956e-4285-9cc2-928ec6492277-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4gtdk\" (UID: \"22d71632-956e-4285-9cc2-928ec6492277\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gtdk"
Sep 17 14:05:01.473892 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.472143    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6504733b-af2d-45a5-a817-7fe57f23d503-host-run-multus-certs\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.473892 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.472198    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6504733b-af2d-45a5-a817-7fe57f23d503-host-run-multus-certs\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.473892 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.472200    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6504733b-af2d-45a5-a817-7fe57f23d503-cni-binary-copy\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.473892 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.472240    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/11334641-b1cc-4ede-aa2f-998edc2a4db0-etc-sysctl-d\") pod \"tuned-6bcj9\" (UID: \"11334641-b1cc-4ede-aa2f-998edc2a4db0\") " pod="openshift-cluster-node-tuning-operator/tuned-6bcj9"
Sep 17 14:05:01.474814 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.472241    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7824d21a-4398-41f6-9303-501f38a2cfd4-node-log\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.474814 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:01.472293    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 14:05:01.474814 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.472327    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7824d21a-4398-41f6-9303-501f38a2cfd4-host-cni-netd\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.474814 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:01.472344    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31e72879-bbfc-4684-bcbb-fd79b4eec4b6-original-pull-secret podName:31e72879-bbfc-4684-bcbb-fd79b4eec4b6 nodeName:}" failed. No retries permitted until 2026-09-17 14:05:01.972326523 +0000 UTC m=+2.973869983 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/31e72879-bbfc-4684-bcbb-fd79b4eec4b6-original-pull-secret") pod "global-pull-secret-syncer-7k2wk" (UID: "31e72879-bbfc-4684-bcbb-fd79b4eec4b6") : object "kube-system"/"original-pull-secret" not registered
Sep 17 14:05:01.474814 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.472291    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/82aa9496-50ec-4a83-9d0c-d227df61037a-cni-binary-copy\") pod \"multus-additional-cni-plugins-xflkb\" (UID: \"82aa9496-50ec-4a83-9d0c-d227df61037a\") " pod="openshift-multus/multus-additional-cni-plugins-xflkb"
Sep 17 14:05:01.474814 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.472367    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvl8x\" (UniqueName: \"kubernetes.io/projected/a2cbe70b-eb96-4779-9143-b95a1204aae2-kube-api-access-nvl8x\") pod \"network-check-target-dg4gz\" (UID: \"a2cbe70b-eb96-4779-9143-b95a1204aae2\") " pod="openshift-network-diagnostics/network-check-target-dg4gz"
Sep 17 14:05:01.474814 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.472419    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/22d71632-956e-4285-9cc2-928ec6492277-etc-selinux\") pod \"aws-ebs-csi-driver-node-4gtdk\" (UID: \"22d71632-956e-4285-9cc2-928ec6492277\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gtdk"
Sep 17 14:05:01.474814 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.472474    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/22d71632-956e-4285-9cc2-928ec6492277-device-dir\") pod \"aws-ebs-csi-driver-node-4gtdk\" (UID: \"22d71632-956e-4285-9cc2-928ec6492277\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gtdk"
Sep 17 14:05:01.474814 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.472505    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8x42m\" (UniqueName: \"kubernetes.io/projected/4462f6aa-ccc5-42d0-8623-f0dad6105560-kube-api-access-8x42m\") pod \"node-ca-72skg\" (UID: \"4462f6aa-ccc5-42d0-8623-f0dad6105560\") " pod="openshift-image-registry/node-ca-72skg"
Sep 17 14:05:01.474814 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.472535    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2mnp\" (UniqueName: \"kubernetes.io/projected/82aa9496-50ec-4a83-9d0c-d227df61037a-kube-api-access-f2mnp\") pod \"multus-additional-cni-plugins-xflkb\" (UID: \"82aa9496-50ec-4a83-9d0c-d227df61037a\") " pod="openshift-multus/multus-additional-cni-plugins-xflkb"
Sep 17 14:05:01.474814 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.472547    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/22d71632-956e-4285-9cc2-928ec6492277-device-dir\") pod \"aws-ebs-csi-driver-node-4gtdk\" (UID: \"22d71632-956e-4285-9cc2-928ec6492277\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gtdk"
Sep 17 14:05:01.474814 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.472562    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/11334641-b1cc-4ede-aa2f-998edc2a4db0-etc-sysctl-conf\") pod \"tuned-6bcj9\" (UID: \"11334641-b1cc-4ede-aa2f-998edc2a4db0\") " pod="openshift-cluster-node-tuning-operator/tuned-6bcj9"
Sep 17 14:05:01.474814 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.472587    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11334641-b1cc-4ede-aa2f-998edc2a4db0-sys\") pod \"tuned-6bcj9\" (UID: \"11334641-b1cc-4ede-aa2f-998edc2a4db0\") " pod="openshift-cluster-node-tuning-operator/tuned-6bcj9"
Sep 17 14:05:01.474814 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.472681    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11334641-b1cc-4ede-aa2f-998edc2a4db0-sys\") pod \"tuned-6bcj9\" (UID: \"11334641-b1cc-4ede-aa2f-998edc2a4db0\") " pod="openshift-cluster-node-tuning-operator/tuned-6bcj9"
Sep 17 14:05:01.474814 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.472726    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/11334641-b1cc-4ede-aa2f-998edc2a4db0-etc-sysctl-conf\") pod \"tuned-6bcj9\" (UID: \"11334641-b1cc-4ede-aa2f-998edc2a4db0\") " pod="openshift-cluster-node-tuning-operator/tuned-6bcj9"
Sep 17 14:05:01.476384 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.476355    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/11334641-b1cc-4ede-aa2f-998edc2a4db0-etc-tuned\") pod \"tuned-6bcj9\" (UID: \"11334641-b1cc-4ede-aa2f-998edc2a4db0\") " pod="openshift-cluster-node-tuning-operator/tuned-6bcj9"
Sep 17 14:05:01.477389 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.477365    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f-agent-certs\") pod \"konnectivity-agent-2j77v\" (UID: \"bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f\") " pod="kube-system/konnectivity-agent-2j77v"
Sep 17 14:05:01.478749 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.478704    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11334641-b1cc-4ede-aa2f-998edc2a4db0-tmp\") pod \"tuned-6bcj9\" (UID: \"11334641-b1cc-4ede-aa2f-998edc2a4db0\") " pod="openshift-cluster-node-tuning-operator/tuned-6bcj9"
Sep 17 14:05:01.478917 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.478893    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7824d21a-4398-41f6-9303-501f38a2cfd4-ovn-node-metrics-cert\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.483424 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.483400    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4t9c\" (UniqueName: \"kubernetes.io/projected/22d71632-956e-4285-9cc2-928ec6492277-kube-api-access-t4t9c\") pod \"aws-ebs-csi-driver-node-4gtdk\" (UID: \"22d71632-956e-4285-9cc2-928ec6492277\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gtdk"
Sep 17 14:05:01.485963 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.485618    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zz6h9\" (UniqueName: \"kubernetes.io/projected/6504733b-af2d-45a5-a817-7fe57f23d503-kube-api-access-zz6h9\") pod \"multus-t4548\" (UID: \"6504733b-af2d-45a5-a817-7fe57f23d503\") " pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.489294 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:01.489268    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 14:05:01.489399 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:01.489302    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 14:05:01.489399 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:01.489315    2582 projected.go:196] Error preparing data for projected volume kube-api-access-nvl8x for pod openshift-network-diagnostics/network-check-target-dg4gz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 14:05:01.489399 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:01.489371    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a2cbe70b-eb96-4779-9143-b95a1204aae2-kube-api-access-nvl8x podName:a2cbe70b-eb96-4779-9143-b95a1204aae2 nodeName:}" failed. No retries permitted until 2026-09-17 14:05:01.989354053 +0000 UTC m=+2.990897540 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-nvl8x" (UniqueName: "kubernetes.io/projected/a2cbe70b-eb96-4779-9143-b95a1204aae2-kube-api-access-nvl8x") pod "network-check-target-dg4gz" (UID: "a2cbe70b-eb96-4779-9143-b95a1204aae2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 14:05:01.489702 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.489657    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thz9t\" (UniqueName: \"kubernetes.io/projected/7824d21a-4398-41f6-9303-501f38a2cfd4-kube-api-access-thz9t\") pod \"ovnkube-node-qn4cf\" (UID: \"7824d21a-4398-41f6-9303-501f38a2cfd4\") " pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.489787 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.489727    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpbsl\" (UniqueName: \"kubernetes.io/projected/11334641-b1cc-4ede-aa2f-998edc2a4db0-kube-api-access-hpbsl\") pod \"tuned-6bcj9\" (UID: \"11334641-b1cc-4ede-aa2f-998edc2a4db0\") " pod="openshift-cluster-node-tuning-operator/tuned-6bcj9"
Sep 17 14:05:01.489915 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.489894    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghlb2\" (UniqueName: \"kubernetes.io/projected/927c3577-20a9-4c54-8c98-0af3bcb37c06-kube-api-access-ghlb2\") pod \"iptables-alerter-jwwq2\" (UID: \"927c3577-20a9-4c54-8c98-0af3bcb37c06\") " pod="openshift-network-operator/iptables-alerter-jwwq2"
Sep 17 14:05:01.490203 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.490182    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66f8r\" (UniqueName: \"kubernetes.io/projected/059afb43-5e5e-4db9-bfa2-d874291c1c3a-kube-api-access-66f8r\") pod \"network-metrics-daemon-r4p4g\" (UID: \"059afb43-5e5e-4db9-bfa2-d874291c1c3a\") " pod="openshift-multus/network-metrics-daemon-r4p4g"
Sep 17 14:05:01.490704 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.490639    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8x42m\" (UniqueName: \"kubernetes.io/projected/4462f6aa-ccc5-42d0-8623-f0dad6105560-kube-api-access-8x42m\") pod \"node-ca-72skg\" (UID: \"4462f6aa-ccc5-42d0-8623-f0dad6105560\") " pod="openshift-image-registry/node-ca-72skg"
Sep 17 14:05:01.492395 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.492326    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2mnp\" (UniqueName: \"kubernetes.io/projected/82aa9496-50ec-4a83-9d0c-d227df61037a-kube-api-access-f2mnp\") pod \"multus-additional-cni-plugins-xflkb\" (UID: \"82aa9496-50ec-4a83-9d0c-d227df61037a\") " pod="openshift-multus/multus-additional-cni-plugins-xflkb"
Sep 17 14:05:01.492517 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.492483    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4czcb\" (UniqueName: \"kubernetes.io/projected/1268d6c2-0f80-41e0-98d5-d72d5c03da1b-kube-api-access-4czcb\") pod \"node-resolver-s2nlk\" (UID: \"1268d6c2-0f80-41e0-98d5-d72d5c03da1b\") " pod="openshift-dns/node-resolver-s2nlk"
Sep 17 14:05:01.494595 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.494490    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-53.ec2.internal" event={"ID":"3ed51660b0e0cea0ccab4eff146b4e99","Type":"ContainerStarted","Data":"fc175b6dc4533024e10c63fd65a2fbc3a971a7170cd3bf481291cc73209bf9c0"}
Sep 17 14:05:01.496314 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.496288    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-53.ec2.internal" event={"ID":"ed4e451484feca11635366f26cff0482","Type":"ContainerStarted","Data":"60214601bf38ea06c3fb9365ac14ae6686aa5303dfad5fb54724941fa5b78bb1"}
Sep 17 14:05:01.652204 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.652011    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:01.659904 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.659875    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2j77v"
Sep 17 14:05:01.666708 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:05:01.664807    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7824d21a_4398_41f6_9303_501f38a2cfd4.slice/crio-640d7b7ac18f3ba1b66913cb0c9d96fce531ee44b24dbdddb44746eb9e6fd001 WatchSource:0}: Error finding container 640d7b7ac18f3ba1b66913cb0c9d96fce531ee44b24dbdddb44746eb9e6fd001: Status 404 returned error can't find the container with id 640d7b7ac18f3ba1b66913cb0c9d96fce531ee44b24dbdddb44746eb9e6fd001
Sep 17 14:05:01.670982 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.670866    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6bcj9"
Sep 17 14:05:01.674303 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:05:01.674269    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc013ec0_cbfe_4335_a5c1_a32d9a3f0a5f.slice/crio-896ed3a9f41c00e06dea64a9527849de39e94685de74a833ed699145c0aaa8ad WatchSource:0}: Error finding container 896ed3a9f41c00e06dea64a9527849de39e94685de74a833ed699145c0aaa8ad: Status 404 returned error can't find the container with id 896ed3a9f41c00e06dea64a9527849de39e94685de74a833ed699145c0aaa8ad
Sep 17 14:05:01.677146 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.677122    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-t4548"
Sep 17 14:05:01.681193 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:05:01.681171    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11334641_b1cc_4ede_aa2f_998edc2a4db0.slice/crio-d0dde6332c23a7cffaf3ccbcd8c8c7c05850ea484089c1d04fb220c1398e5f54 WatchSource:0}: Error finding container d0dde6332c23a7cffaf3ccbcd8c8c7c05850ea484089c1d04fb220c1398e5f54: Status 404 returned error can't find the container with id d0dde6332c23a7cffaf3ccbcd8c8c7c05850ea484089c1d04fb220c1398e5f54
Sep 17 14:05:01.685506 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.685478    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jwwq2"
Sep 17 14:05:01.686497 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:05:01.686474    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6504733b_af2d_45a5_a817_7fe57f23d503.slice/crio-097c8093ba2116dc49e6a027cae0ee5b8bde1f1d7f3647482b2aa30c5860d403 WatchSource:0}: Error finding container 097c8093ba2116dc49e6a027cae0ee5b8bde1f1d7f3647482b2aa30c5860d403: Status 404 returned error can't find the container with id 097c8093ba2116dc49e6a027cae0ee5b8bde1f1d7f3647482b2aa30c5860d403
Sep 17 14:05:01.693196 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.693175    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gtdk"
Sep 17 14:05:01.694307 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:05:01.694278    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod927c3577_20a9_4c54_8c98_0af3bcb37c06.slice/crio-ced6c4384801271b0e55a24bcb03258c3688fd7b8d9b9bdc64bb2c959ba20a20 WatchSource:0}: Error finding container ced6c4384801271b0e55a24bcb03258c3688fd7b8d9b9bdc64bb2c959ba20a20: Status 404 returned error can't find the container with id ced6c4384801271b0e55a24bcb03258c3688fd7b8d9b9bdc64bb2c959ba20a20
Sep 17 14:05:01.699607 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.699587    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-s2nlk"
Sep 17 14:05:01.702619 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:05:01.702573    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22d71632_956e_4285_9cc2_928ec6492277.slice/crio-ed4ecb65133109980f886ede5dbd927cb6d12b5c3c3c5c9096112b046c7c1065 WatchSource:0}: Error finding container ed4ecb65133109980f886ede5dbd927cb6d12b5c3c3c5c9096112b046c7c1065: Status 404 returned error can't find the container with id ed4ecb65133109980f886ede5dbd927cb6d12b5c3c3c5c9096112b046c7c1065
Sep 17 14:05:01.706632 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.706548    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-72skg"
Sep 17 14:05:01.708203 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:05:01.708177    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1268d6c2_0f80_41e0_98d5_d72d5c03da1b.slice/crio-18aadce9222dd5b178122241a0e8bb26c3cccc45741ce417dbd90d715c5fdb15 WatchSource:0}: Error finding container 18aadce9222dd5b178122241a0e8bb26c3cccc45741ce417dbd90d715c5fdb15: Status 404 returned error can't find the container with id 18aadce9222dd5b178122241a0e8bb26c3cccc45741ce417dbd90d715c5fdb15
Sep 17 14:05:01.712350 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.712309    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xflkb"
Sep 17 14:05:01.716088 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:05:01.716062    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4462f6aa_ccc5_42d0_8623_f0dad6105560.slice/crio-89822e605c7c5515782ccfb288e0fe10f35f26b5a0c8d7d9460141b1c9cf0fa1 WatchSource:0}: Error finding container 89822e605c7c5515782ccfb288e0fe10f35f26b5a0c8d7d9460141b1c9cf0fa1: Status 404 returned error can't find the container with id 89822e605c7c5515782ccfb288e0fe10f35f26b5a0c8d7d9460141b1c9cf0fa1
Sep 17 14:05:01.721564 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:05:01.721539    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82aa9496_50ec_4a83_9d0c_d227df61037a.slice/crio-b14d69c10dffd3c97f16ae34158a290d34cc89555b9f4ab8f0f95f6f5d189a61 WatchSource:0}: Error finding container b14d69c10dffd3c97f16ae34158a290d34cc89555b9f4ab8f0f95f6f5d189a61: Status 404 returned error can't find the container with id b14d69c10dffd3c97f16ae34158a290d34cc89555b9f4ab8f0f95f6f5d189a61
Sep 17 14:05:01.977219 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.977123    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/059afb43-5e5e-4db9-bfa2-d874291c1c3a-metrics-certs\") pod \"network-metrics-daemon-r4p4g\" (UID: \"059afb43-5e5e-4db9-bfa2-d874291c1c3a\") " pod="openshift-multus/network-metrics-daemon-r4p4g"
Sep 17 14:05:01.977219 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:01.977189    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31e72879-bbfc-4684-bcbb-fd79b4eec4b6-original-pull-secret\") pod \"global-pull-secret-syncer-7k2wk\" (UID: \"31e72879-bbfc-4684-bcbb-fd79b4eec4b6\") " pod="kube-system/global-pull-secret-syncer-7k2wk"
Sep 17 14:05:01.977449 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:01.977298    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 14:05:01.977449 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:01.977315    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 14:05:01.977449 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:01.977375    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/059afb43-5e5e-4db9-bfa2-d874291c1c3a-metrics-certs podName:059afb43-5e5e-4db9-bfa2-d874291c1c3a nodeName:}" failed. No retries permitted until 2026-09-17 14:05:02.977351973 +0000 UTC m=+3.978895437 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/059afb43-5e5e-4db9-bfa2-d874291c1c3a-metrics-certs") pod "network-metrics-daemon-r4p4g" (UID: "059afb43-5e5e-4db9-bfa2-d874291c1c3a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 14:05:01.977449 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:01.977397    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31e72879-bbfc-4684-bcbb-fd79b4eec4b6-original-pull-secret podName:31e72879-bbfc-4684-bcbb-fd79b4eec4b6 nodeName:}" failed. No retries permitted until 2026-09-17 14:05:02.977385641 +0000 UTC m=+3.978929106 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/31e72879-bbfc-4684-bcbb-fd79b4eec4b6-original-pull-secret") pod "global-pull-secret-syncer-7k2wk" (UID: "31e72879-bbfc-4684-bcbb-fd79b4eec4b6") : object "kube-system"/"original-pull-secret" not registered
Sep 17 14:05:02.077868 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:02.077827    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvl8x\" (UniqueName: \"kubernetes.io/projected/a2cbe70b-eb96-4779-9143-b95a1204aae2-kube-api-access-nvl8x\") pod \"network-check-target-dg4gz\" (UID: \"a2cbe70b-eb96-4779-9143-b95a1204aae2\") " pod="openshift-network-diagnostics/network-check-target-dg4gz"
Sep 17 14:05:02.078060 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:02.078016    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 14:05:02.078060 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:02.078045    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 14:05:02.078060 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:02.078058    2582 projected.go:196] Error preparing data for projected volume kube-api-access-nvl8x for pod openshift-network-diagnostics/network-check-target-dg4gz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 14:05:02.078298 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:02.078127    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a2cbe70b-eb96-4779-9143-b95a1204aae2-kube-api-access-nvl8x podName:a2cbe70b-eb96-4779-9143-b95a1204aae2 nodeName:}" failed. No retries permitted until 2026-09-17 14:05:03.078107326 +0000 UTC m=+4.079650825 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-nvl8x" (UniqueName: "kubernetes.io/projected/a2cbe70b-eb96-4779-9143-b95a1204aae2-kube-api-access-nvl8x") pod "network-check-target-dg4gz" (UID: "a2cbe70b-eb96-4779-9143-b95a1204aae2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 14:05:02.465818 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:02.465727    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 14:00:01 +0000 UTC" deadline="2028-04-04 00:49:22.298942299 +0000 UTC"
Sep 17 14:05:02.465818 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:02.465759    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13546h44m19.833186829s"
Sep 17 14:05:02.487474 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:02.487433    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dg4gz"
Sep 17 14:05:02.487686 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:02.487569    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dg4gz" podUID="a2cbe70b-eb96-4779-9143-b95a1204aae2"
Sep 17 14:05:02.499787 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:02.499736    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xflkb" event={"ID":"82aa9496-50ec-4a83-9d0c-d227df61037a","Type":"ContainerStarted","Data":"b14d69c10dffd3c97f16ae34158a290d34cc89555b9f4ab8f0f95f6f5d189a61"}
Sep 17 14:05:02.501098 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:02.501068    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-72skg" event={"ID":"4462f6aa-ccc5-42d0-8623-f0dad6105560","Type":"ContainerStarted","Data":"89822e605c7c5515782ccfb288e0fe10f35f26b5a0c8d7d9460141b1c9cf0fa1"}
Sep 17 14:05:02.502420 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:02.502382    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gtdk" event={"ID":"22d71632-956e-4285-9cc2-928ec6492277","Type":"ContainerStarted","Data":"ed4ecb65133109980f886ede5dbd927cb6d12b5c3c3c5c9096112b046c7c1065"}
Sep 17 14:05:02.503775 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:02.503753    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-t4548" event={"ID":"6504733b-af2d-45a5-a817-7fe57f23d503","Type":"ContainerStarted","Data":"097c8093ba2116dc49e6a027cae0ee5b8bde1f1d7f3647482b2aa30c5860d403"}
Sep 17 14:05:02.505181 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:02.505154    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6bcj9" event={"ID":"11334641-b1cc-4ede-aa2f-998edc2a4db0","Type":"ContainerStarted","Data":"d0dde6332c23a7cffaf3ccbcd8c8c7c05850ea484089c1d04fb220c1398e5f54"}
Sep 17 14:05:02.506483 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:02.506463    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2j77v" event={"ID":"bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f","Type":"ContainerStarted","Data":"896ed3a9f41c00e06dea64a9527849de39e94685de74a833ed699145c0aaa8ad"}
Sep 17 14:05:02.508985 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:02.508961    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf" event={"ID":"7824d21a-4398-41f6-9303-501f38a2cfd4","Type":"ContainerStarted","Data":"640d7b7ac18f3ba1b66913cb0c9d96fce531ee44b24dbdddb44746eb9e6fd001"}
Sep 17 14:05:02.510350 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:02.510326    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-s2nlk" event={"ID":"1268d6c2-0f80-41e0-98d5-d72d5c03da1b","Type":"ContainerStarted","Data":"18aadce9222dd5b178122241a0e8bb26c3cccc45741ce417dbd90d715c5fdb15"}
Sep 17 14:05:02.511660 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:02.511628    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jwwq2" event={"ID":"927c3577-20a9-4c54-8c98-0af3bcb37c06","Type":"ContainerStarted","Data":"ced6c4384801271b0e55a24bcb03258c3688fd7b8d9b9bdc64bb2c959ba20a20"}
Sep 17 14:05:02.986755 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:02.986011    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31e72879-bbfc-4684-bcbb-fd79b4eec4b6-original-pull-secret\") pod \"global-pull-secret-syncer-7k2wk\" (UID: \"31e72879-bbfc-4684-bcbb-fd79b4eec4b6\") " pod="kube-system/global-pull-secret-syncer-7k2wk"
Sep 17 14:05:02.986755 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:02.986085    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/059afb43-5e5e-4db9-bfa2-d874291c1c3a-metrics-certs\") pod \"network-metrics-daemon-r4p4g\" (UID: \"059afb43-5e5e-4db9-bfa2-d874291c1c3a\") " pod="openshift-multus/network-metrics-daemon-r4p4g"
Sep 17 14:05:02.986755 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:02.986255    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 14:05:02.986755 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:02.986315    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/059afb43-5e5e-4db9-bfa2-d874291c1c3a-metrics-certs podName:059afb43-5e5e-4db9-bfa2-d874291c1c3a nodeName:}" failed. No retries permitted until 2026-09-17 14:05:04.986295586 +0000 UTC m=+5.987839044 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/059afb43-5e5e-4db9-bfa2-d874291c1c3a-metrics-certs") pod "network-metrics-daemon-r4p4g" (UID: "059afb43-5e5e-4db9-bfa2-d874291c1c3a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 14:05:02.986755 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:02.986732    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 14:05:02.987201 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:02.986786    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31e72879-bbfc-4684-bcbb-fd79b4eec4b6-original-pull-secret podName:31e72879-bbfc-4684-bcbb-fd79b4eec4b6 nodeName:}" failed. No retries permitted until 2026-09-17 14:05:04.986771131 +0000 UTC m=+5.988314597 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/31e72879-bbfc-4684-bcbb-fd79b4eec4b6-original-pull-secret") pod "global-pull-secret-syncer-7k2wk" (UID: "31e72879-bbfc-4684-bcbb-fd79b4eec4b6") : object "kube-system"/"original-pull-secret" not registered
Sep 17 14:05:03.087717 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:03.087098    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvl8x\" (UniqueName: \"kubernetes.io/projected/a2cbe70b-eb96-4779-9143-b95a1204aae2-kube-api-access-nvl8x\") pod \"network-check-target-dg4gz\" (UID: \"a2cbe70b-eb96-4779-9143-b95a1204aae2\") " pod="openshift-network-diagnostics/network-check-target-dg4gz"
Sep 17 14:05:03.087717 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:03.087257    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 14:05:03.087717 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:03.087276    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 14:05:03.087717 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:03.087290    2582 projected.go:196] Error preparing data for projected volume kube-api-access-nvl8x for pod openshift-network-diagnostics/network-check-target-dg4gz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 14:05:03.087717 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:03.087347    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a2cbe70b-eb96-4779-9143-b95a1204aae2-kube-api-access-nvl8x podName:a2cbe70b-eb96-4779-9143-b95a1204aae2 nodeName:}" failed. No retries permitted until 2026-09-17 14:05:05.087327951 +0000 UTC m=+6.088871411 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-nvl8x" (UniqueName: "kubernetes.io/projected/a2cbe70b-eb96-4779-9143-b95a1204aae2-kube-api-access-nvl8x") pod "network-check-target-dg4gz" (UID: "a2cbe70b-eb96-4779-9143-b95a1204aae2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 14:05:03.466673 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:03.466604    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 14:00:01 +0000 UTC" deadline="2028-03-01 14:12:00.794834325 +0000 UTC"
Sep 17 14:05:03.466673 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:03.466665    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12744h6m57.328175112s"
Sep 17 14:05:03.488914 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:03.488593    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4p4g"
Sep 17 14:05:03.488914 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:03.488812    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4p4g" podUID="059afb43-5e5e-4db9-bfa2-d874291c1c3a"
Sep 17 14:05:03.488914 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:03.488833    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7k2wk"
Sep 17 14:05:03.489187 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:03.488920    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7k2wk" podUID="31e72879-bbfc-4684-bcbb-fd79b4eec4b6"
Sep 17 14:05:03.521278 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:03.520452    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-53.ec2.internal" event={"ID":"3ed51660b0e0cea0ccab4eff146b4e99","Type":"ContainerStarted","Data":"3df18ad7cdd4b47d5c4a7a29216034f759d61418ea46852d74e86b9efdf0e99d"}
Sep 17 14:05:04.487946 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:04.487819    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dg4gz"
Sep 17 14:05:04.488403 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:04.487947    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dg4gz" podUID="a2cbe70b-eb96-4779-9143-b95a1204aae2"
Sep 17 14:05:04.545553 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:04.545515    2582 generic.go:356] "Generic (PLEG): container finished" podID="ed4e451484feca11635366f26cff0482" containerID="9ca8c35daebbb3ea36bf657fcd855ce7691ef97b8813e25bdc6f7712cf777221" exitCode=0
Sep 17 14:05:04.545729 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:04.545643    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-53.ec2.internal" event={"ID":"ed4e451484feca11635366f26cff0482","Type":"ContainerDied","Data":"9ca8c35daebbb3ea36bf657fcd855ce7691ef97b8813e25bdc6f7712cf777221"}
Sep 17 14:05:04.566985 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:04.566931    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-53.ec2.internal" podStartSLOduration=4.566909327 podStartE2EDuration="4.566909327s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 14:05:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:05:03.540579503 +0000 UTC m=+4.542122983" watchObservedRunningTime="2026-09-17 14:05:04.566909327 +0000 UTC m=+5.568452799"
Sep 17 14:05:05.001828 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:05.001701    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31e72879-bbfc-4684-bcbb-fd79b4eec4b6-original-pull-secret\") pod \"global-pull-secret-syncer-7k2wk\" (UID: \"31e72879-bbfc-4684-bcbb-fd79b4eec4b6\") " pod="kube-system/global-pull-secret-syncer-7k2wk"
Sep 17 14:05:05.001828 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:05.001780    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/059afb43-5e5e-4db9-bfa2-d874291c1c3a-metrics-certs\") pod \"network-metrics-daemon-r4p4g\" (UID: \"059afb43-5e5e-4db9-bfa2-d874291c1c3a\") " pod="openshift-multus/network-metrics-daemon-r4p4g"
Sep 17 14:05:05.002059 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:05.001909    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 14:05:05.002059 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:05.001973    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/059afb43-5e5e-4db9-bfa2-d874291c1c3a-metrics-certs podName:059afb43-5e5e-4db9-bfa2-d874291c1c3a nodeName:}" failed. No retries permitted until 2026-09-17 14:05:09.001952469 +0000 UTC m=+10.003495945 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/059afb43-5e5e-4db9-bfa2-d874291c1c3a-metrics-certs") pod "network-metrics-daemon-r4p4g" (UID: "059afb43-5e5e-4db9-bfa2-d874291c1c3a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 14:05:05.002382 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:05.002360    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 14:05:05.002473 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:05.002408    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31e72879-bbfc-4684-bcbb-fd79b4eec4b6-original-pull-secret podName:31e72879-bbfc-4684-bcbb-fd79b4eec4b6 nodeName:}" failed. No retries permitted until 2026-09-17 14:05:09.002393426 +0000 UTC m=+10.003936878 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/31e72879-bbfc-4684-bcbb-fd79b4eec4b6-original-pull-secret") pod "global-pull-secret-syncer-7k2wk" (UID: "31e72879-bbfc-4684-bcbb-fd79b4eec4b6") : object "kube-system"/"original-pull-secret" not registered
Sep 17 14:05:05.102982 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:05.102947    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvl8x\" (UniqueName: \"kubernetes.io/projected/a2cbe70b-eb96-4779-9143-b95a1204aae2-kube-api-access-nvl8x\") pod \"network-check-target-dg4gz\" (UID: \"a2cbe70b-eb96-4779-9143-b95a1204aae2\") " pod="openshift-network-diagnostics/network-check-target-dg4gz"
Sep 17 14:05:05.103153 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:05.103140    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 14:05:05.103213 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:05.103159    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 14:05:05.103213 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:05.103174    2582 projected.go:196] Error preparing data for projected volume kube-api-access-nvl8x for pod openshift-network-diagnostics/network-check-target-dg4gz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 14:05:05.103335 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:05.103232    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a2cbe70b-eb96-4779-9143-b95a1204aae2-kube-api-access-nvl8x podName:a2cbe70b-eb96-4779-9143-b95a1204aae2 nodeName:}" failed. No retries permitted until 2026-09-17 14:05:09.103214427 +0000 UTC m=+10.104757880 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-nvl8x" (UniqueName: "kubernetes.io/projected/a2cbe70b-eb96-4779-9143-b95a1204aae2-kube-api-access-nvl8x") pod "network-check-target-dg4gz" (UID: "a2cbe70b-eb96-4779-9143-b95a1204aae2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 14:05:05.487573 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:05.487496    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7k2wk"
Sep 17 14:05:05.487742 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:05.487660    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7k2wk" podUID="31e72879-bbfc-4684-bcbb-fd79b4eec4b6"
Sep 17 14:05:05.487800 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:05.487748    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4p4g"
Sep 17 14:05:05.487963 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:05.487874    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4p4g" podUID="059afb43-5e5e-4db9-bfa2-d874291c1c3a"
Sep 17 14:05:05.556080 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:05.556032    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-53.ec2.internal" event={"ID":"ed4e451484feca11635366f26cff0482","Type":"ContainerStarted","Data":"f9a3684314367f8a780273c978d133f552626a95003e071698dc8e2c0c1d538e"}
Sep 17 14:05:05.575178 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:05.575117    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-53.ec2.internal" podStartSLOduration=5.575096287 podStartE2EDuration="5.575096287s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 14:05:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:05:05.574325756 +0000 UTC m=+6.575869228" watchObservedRunningTime="2026-09-17 14:05:05.575096287 +0000 UTC m=+6.576639761"
Sep 17 14:05:06.487353 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:06.486911    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dg4gz"
Sep 17 14:05:06.487353 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:06.487043    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dg4gz" podUID="a2cbe70b-eb96-4779-9143-b95a1204aae2"
Sep 17 14:05:07.487474 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:07.487437    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4p4g"
Sep 17 14:05:07.487959 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:07.487665    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4p4g" podUID="059afb43-5e5e-4db9-bfa2-d874291c1c3a"
Sep 17 14:05:07.487959 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:07.487673    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7k2wk"
Sep 17 14:05:07.487959 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:07.487774    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7k2wk" podUID="31e72879-bbfc-4684-bcbb-fd79b4eec4b6"
Sep 17 14:05:08.487897 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:08.487816    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dg4gz"
Sep 17 14:05:08.488345 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:08.487965    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dg4gz" podUID="a2cbe70b-eb96-4779-9143-b95a1204aae2"
Sep 17 14:05:09.035519 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:09.035480    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/059afb43-5e5e-4db9-bfa2-d874291c1c3a-metrics-certs\") pod \"network-metrics-daemon-r4p4g\" (UID: \"059afb43-5e5e-4db9-bfa2-d874291c1c3a\") " pod="openshift-multus/network-metrics-daemon-r4p4g"
Sep 17 14:05:09.035732 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:09.035646    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31e72879-bbfc-4684-bcbb-fd79b4eec4b6-original-pull-secret\") pod \"global-pull-secret-syncer-7k2wk\" (UID: \"31e72879-bbfc-4684-bcbb-fd79b4eec4b6\") " pod="kube-system/global-pull-secret-syncer-7k2wk"
Sep 17 14:05:09.035732 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:09.035678    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 14:05:09.035870 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:09.035753    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/059afb43-5e5e-4db9-bfa2-d874291c1c3a-metrics-certs podName:059afb43-5e5e-4db9-bfa2-d874291c1c3a nodeName:}" failed. No retries permitted until 2026-09-17 14:05:17.035732538 +0000 UTC m=+18.037276001 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/059afb43-5e5e-4db9-bfa2-d874291c1c3a-metrics-certs") pod "network-metrics-daemon-r4p4g" (UID: "059afb43-5e5e-4db9-bfa2-d874291c1c3a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 14:05:09.035870 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:09.035763    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 14:05:09.035870 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:09.035808    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31e72879-bbfc-4684-bcbb-fd79b4eec4b6-original-pull-secret podName:31e72879-bbfc-4684-bcbb-fd79b4eec4b6 nodeName:}" failed. No retries permitted until 2026-09-17 14:05:17.035798199 +0000 UTC m=+18.037341651 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/31e72879-bbfc-4684-bcbb-fd79b4eec4b6-original-pull-secret") pod "global-pull-secret-syncer-7k2wk" (UID: "31e72879-bbfc-4684-bcbb-fd79b4eec4b6") : object "kube-system"/"original-pull-secret" not registered
Sep 17 14:05:09.136799 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:09.136169    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvl8x\" (UniqueName: \"kubernetes.io/projected/a2cbe70b-eb96-4779-9143-b95a1204aae2-kube-api-access-nvl8x\") pod \"network-check-target-dg4gz\" (UID: \"a2cbe70b-eb96-4779-9143-b95a1204aae2\") " pod="openshift-network-diagnostics/network-check-target-dg4gz"
Sep 17 14:05:09.136799 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:09.136354    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 14:05:09.136799 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:09.136374    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 14:05:09.136799 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:09.136386    2582 projected.go:196] Error preparing data for projected volume kube-api-access-nvl8x for pod openshift-network-diagnostics/network-check-target-dg4gz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 14:05:09.136799 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:09.136443    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a2cbe70b-eb96-4779-9143-b95a1204aae2-kube-api-access-nvl8x podName:a2cbe70b-eb96-4779-9143-b95a1204aae2 nodeName:}" failed. No retries permitted until 2026-09-17 14:05:17.13642533 +0000 UTC m=+18.137968797 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-nvl8x" (UniqueName: "kubernetes.io/projected/a2cbe70b-eb96-4779-9143-b95a1204aae2-kube-api-access-nvl8x") pod "network-check-target-dg4gz" (UID: "a2cbe70b-eb96-4779-9143-b95a1204aae2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 14:05:09.488679 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:09.488584    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4p4g"
Sep 17 14:05:09.489632 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:09.489241    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4p4g" podUID="059afb43-5e5e-4db9-bfa2-d874291c1c3a"
Sep 17 14:05:09.489632 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:09.488663    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7k2wk"
Sep 17 14:05:09.489632 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:09.489462    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7k2wk" podUID="31e72879-bbfc-4684-bcbb-fd79b4eec4b6"
Sep 17 14:05:10.487413 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:10.487370    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dg4gz"
Sep 17 14:05:10.487572 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:10.487515    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dg4gz" podUID="a2cbe70b-eb96-4779-9143-b95a1204aae2"
Sep 17 14:05:11.487783 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:11.487750    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4p4g"
Sep 17 14:05:11.488254 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:11.487865    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4p4g" podUID="059afb43-5e5e-4db9-bfa2-d874291c1c3a"
Sep 17 14:05:11.488254 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:11.487914    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7k2wk"
Sep 17 14:05:11.488254 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:11.488004    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7k2wk" podUID="31e72879-bbfc-4684-bcbb-fd79b4eec4b6"
Sep 17 14:05:12.487151 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:12.487115    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dg4gz"
Sep 17 14:05:12.487346 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:12.487246    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dg4gz" podUID="a2cbe70b-eb96-4779-9143-b95a1204aae2"
Sep 17 14:05:13.487266 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:13.487195    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4p4g"
Sep 17 14:05:13.487727 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:13.487350    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4p4g" podUID="059afb43-5e5e-4db9-bfa2-d874291c1c3a"
Sep 17 14:05:13.487727 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:13.487363    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7k2wk"
Sep 17 14:05:13.487727 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:13.487470    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7k2wk" podUID="31e72879-bbfc-4684-bcbb-fd79b4eec4b6"
Sep 17 14:05:14.487244 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:14.487210    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dg4gz"
Sep 17 14:05:14.487428 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:14.487339    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dg4gz" podUID="a2cbe70b-eb96-4779-9143-b95a1204aae2"
Sep 17 14:05:15.487186 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:15.487147    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7k2wk"
Sep 17 14:05:15.487360 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:15.487147    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4p4g"
Sep 17 14:05:15.487360 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:15.487284    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7k2wk" podUID="31e72879-bbfc-4684-bcbb-fd79b4eec4b6"
Sep 17 14:05:15.487461 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:15.487368    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4p4g" podUID="059afb43-5e5e-4db9-bfa2-d874291c1c3a"
Sep 17 14:05:16.487112 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:16.487069    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dg4gz"
Sep 17 14:05:16.487311 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:16.487202    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dg4gz" podUID="a2cbe70b-eb96-4779-9143-b95a1204aae2"
Sep 17 14:05:17.094301 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:17.094254    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/059afb43-5e5e-4db9-bfa2-d874291c1c3a-metrics-certs\") pod \"network-metrics-daemon-r4p4g\" (UID: \"059afb43-5e5e-4db9-bfa2-d874291c1c3a\") " pod="openshift-multus/network-metrics-daemon-r4p4g"
Sep 17 14:05:17.094761 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:17.094335    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31e72879-bbfc-4684-bcbb-fd79b4eec4b6-original-pull-secret\") pod \"global-pull-secret-syncer-7k2wk\" (UID: \"31e72879-bbfc-4684-bcbb-fd79b4eec4b6\") " pod="kube-system/global-pull-secret-syncer-7k2wk"
Sep 17 14:05:17.094761 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:17.094423    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 14:05:17.094761 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:17.094464    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 14:05:17.094761 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:17.094498    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/059afb43-5e5e-4db9-bfa2-d874291c1c3a-metrics-certs podName:059afb43-5e5e-4db9-bfa2-d874291c1c3a nodeName:}" failed. No retries permitted until 2026-09-17 14:05:33.094482157 +0000 UTC m=+34.096025626 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/059afb43-5e5e-4db9-bfa2-d874291c1c3a-metrics-certs") pod "network-metrics-daemon-r4p4g" (UID: "059afb43-5e5e-4db9-bfa2-d874291c1c3a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 14:05:17.094761 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:17.094519    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31e72879-bbfc-4684-bcbb-fd79b4eec4b6-original-pull-secret podName:31e72879-bbfc-4684-bcbb-fd79b4eec4b6 nodeName:}" failed. No retries permitted until 2026-09-17 14:05:33.094509023 +0000 UTC m=+34.096052477 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/31e72879-bbfc-4684-bcbb-fd79b4eec4b6-original-pull-secret") pod "global-pull-secret-syncer-7k2wk" (UID: "31e72879-bbfc-4684-bcbb-fd79b4eec4b6") : object "kube-system"/"original-pull-secret" not registered
Sep 17 14:05:17.194815 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:17.194778    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvl8x\" (UniqueName: \"kubernetes.io/projected/a2cbe70b-eb96-4779-9143-b95a1204aae2-kube-api-access-nvl8x\") pod \"network-check-target-dg4gz\" (UID: \"a2cbe70b-eb96-4779-9143-b95a1204aae2\") " pod="openshift-network-diagnostics/network-check-target-dg4gz"
Sep 17 14:05:17.194977 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:17.194920    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 14:05:17.194977 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:17.194941    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 14:05:17.194977 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:17.194950    2582 projected.go:196] Error preparing data for projected volume kube-api-access-nvl8x for pod openshift-network-diagnostics/network-check-target-dg4gz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 14:05:17.195101 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:17.195005    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a2cbe70b-eb96-4779-9143-b95a1204aae2-kube-api-access-nvl8x podName:a2cbe70b-eb96-4779-9143-b95a1204aae2 nodeName:}" failed. No retries permitted until 2026-09-17 14:05:33.194986714 +0000 UTC m=+34.196530179 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-nvl8x" (UniqueName: "kubernetes.io/projected/a2cbe70b-eb96-4779-9143-b95a1204aae2-kube-api-access-nvl8x") pod "network-check-target-dg4gz" (UID: "a2cbe70b-eb96-4779-9143-b95a1204aae2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 14:05:17.487114 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:17.487018    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7k2wk"
Sep 17 14:05:17.487272 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:17.487172    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7k2wk" podUID="31e72879-bbfc-4684-bcbb-fd79b4eec4b6"
Sep 17 14:05:17.487272 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:17.487223    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4p4g"
Sep 17 14:05:17.487382 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:17.487340    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4p4g" podUID="059afb43-5e5e-4db9-bfa2-d874291c1c3a"
Sep 17 14:05:18.486945 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:18.486908    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dg4gz"
Sep 17 14:05:18.487358 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:18.487026    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dg4gz" podUID="a2cbe70b-eb96-4779-9143-b95a1204aae2"
Sep 17 14:05:19.487947 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:19.487909    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4p4g"
Sep 17 14:05:19.488334 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:19.488040    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4p4g" podUID="059afb43-5e5e-4db9-bfa2-d874291c1c3a"
Sep 17 14:05:19.488334 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:19.488100    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7k2wk"
Sep 17 14:05:19.488334 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:19.488212    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7k2wk" podUID="31e72879-bbfc-4684-bcbb-fd79b4eec4b6"
Sep 17 14:05:20.487906 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:20.487736    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dg4gz"
Sep 17 14:05:20.487993 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:20.487975    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dg4gz" podUID="a2cbe70b-eb96-4779-9143-b95a1204aae2"
Sep 17 14:05:20.588001 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:20.587977    2582 generic.go:356] "Generic (PLEG): container finished" podID="82aa9496-50ec-4a83-9d0c-d227df61037a" containerID="8a21edc33621e27c890e040610305ff2c35ea3745fadd0625262cbd81caf8675" exitCode=0
Sep 17 14:05:20.588079 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:20.588047    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xflkb" event={"ID":"82aa9496-50ec-4a83-9d0c-d227df61037a","Type":"ContainerDied","Data":"8a21edc33621e27c890e040610305ff2c35ea3745fadd0625262cbd81caf8675"}
Sep 17 14:05:20.589403 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:20.589347    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-72skg" event={"ID":"4462f6aa-ccc5-42d0-8623-f0dad6105560","Type":"ContainerStarted","Data":"d9d161fccab53baceb43c4f7ecba01b0f4462c4fa2aee6ac39edf8bde84cfa07"}
Sep 17 14:05:20.590878 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:20.590860    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gtdk" event={"ID":"22d71632-956e-4285-9cc2-928ec6492277","Type":"ContainerStarted","Data":"1939608c98e93572eb637b63560e560dd450ae6d2de8ae8ba680e51ed317d22a"}
Sep 17 14:05:20.592289 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:20.592269    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-t4548" event={"ID":"6504733b-af2d-45a5-a817-7fe57f23d503","Type":"ContainerStarted","Data":"acf305d8513e4c50e7bfb3ac9873a6888579a0c290221e69833cac3498617bff"}
Sep 17 14:05:20.593813 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:20.593792    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6bcj9" event={"ID":"11334641-b1cc-4ede-aa2f-998edc2a4db0","Type":"ContainerStarted","Data":"07b80371475db226389f371dceb329ffbe0194d06cb7c523c8c193cc130efe4e"}
Sep 17 14:05:20.595169 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:20.595147    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2j77v" event={"ID":"bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f","Type":"ContainerStarted","Data":"a96e380a7a66b17e612f315b0a4bdd422495a313a619e22c092837de898b98bc"}
Sep 17 14:05:20.597601 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:20.597580    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf" event={"ID":"7824d21a-4398-41f6-9303-501f38a2cfd4","Type":"ContainerStarted","Data":"97efea08bfc3de4bd20a7aee91272087281911d2a3453fdad4b636320bd28abb"}
Sep 17 14:05:20.597695 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:20.597653    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf" event={"ID":"7824d21a-4398-41f6-9303-501f38a2cfd4","Type":"ContainerStarted","Data":"82a60b232182861a7454af740ce052808a56c35349f71971821103937b666d72"}
Sep 17 14:05:20.597741 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:20.597695    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf" event={"ID":"7824d21a-4398-41f6-9303-501f38a2cfd4","Type":"ContainerStarted","Data":"0c1a49a0b721ff937e621aaf058c31e78508e136a0ab5a4cea4028761a231b89"}
Sep 17 14:05:20.597741 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:20.597735    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf" event={"ID":"7824d21a-4398-41f6-9303-501f38a2cfd4","Type":"ContainerStarted","Data":"e361ffaa6aa0350cdc81466ece3cc723f590b207ff766cefd230941b9c8fe9ca"}
Sep 17 14:05:20.597824 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:20.597764    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf" event={"ID":"7824d21a-4398-41f6-9303-501f38a2cfd4","Type":"ContainerStarted","Data":"9c1d575dcc800cdf58a2f77c19809bb7b7b5c77c833a59c8808586954ab00483"}
Sep 17 14:05:20.598966 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:20.598949    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-s2nlk" event={"ID":"1268d6c2-0f80-41e0-98d5-d72d5c03da1b","Type":"ContainerStarted","Data":"d17dd5894f0680737660fb63ca806407365f67d6153427c713beb9a0d1d7036a"}
Sep 17 14:05:20.625499 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:20.625460    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-s2nlk" podStartSLOduration=6.055933692 podStartE2EDuration="20.625446379s" totalImagesPullingTime="14.569512687s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 14:05:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:05:20.625130705 +0000 UTC m=+21.626674178" watchObservedRunningTime="2026-09-17 14:05:20.625446379 +0000 UTC m=+21.626989852"
Sep 17 14:05:20.646805 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:20.646766    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-72skg" podStartSLOduration=2.4131093359999998 podStartE2EDuration="20.646754579s" totalImagesPullingTime="18.233645243s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 14:05:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:05:20.646253695 +0000 UTC m=+21.647797172" watchObservedRunningTime="2026-09-17 14:05:20.646754579 +0000 UTC m=+21.648298054"
Sep 17 14:05:20.661218 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:20.661182    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-2j77v" podStartSLOduration=2.401686671 podStartE2EDuration="20.661169161s" totalImagesPullingTime="18.25948249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:05:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:05:20.660958214 +0000 UTC m=+21.662501689" watchObservedRunningTime="2026-09-17 14:05:20.661169161 +0000 UTC m=+21.662712634"
Sep 17 14:05:20.678186 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:20.678152    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-6bcj9" podStartSLOduration=2.407911412 podStartE2EDuration="20.67813995s" totalImagesPullingTime="18.270228538s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 14:05:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:05:20.677748399 +0000 UTC m=+21.679291875" watchObservedRunningTime="2026-09-17 14:05:20.67813995 +0000 UTC m=+21.679683424"
Sep 17 14:05:20.706715 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:20.706664    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-t4548" podStartSLOduration=2.295888271 podStartE2EDuration="20.706648809s" totalImagesPullingTime="18.410760538s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 14:05:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:05:20.704074105 +0000 UTC m=+21.705617578" watchObservedRunningTime="2026-09-17 14:05:20.706648809 +0000 UTC m=+21.708192283"
Sep 17 14:05:21.487159 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:21.487134    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7k2wk"
Sep 17 14:05:21.487270 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:21.487139    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4p4g"
Sep 17 14:05:21.487270 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:21.487257    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7k2wk" podUID="31e72879-bbfc-4684-bcbb-fd79b4eec4b6"
Sep 17 14:05:21.487345 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:21.487305    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4p4g" podUID="059afb43-5e5e-4db9-bfa2-d874291c1c3a"
Sep 17 14:05:21.524885 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:21.524859    2582 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 17 14:05:21.565170 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:21.565070    2582 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-17T14:05:21.524882531Z","UUID":"db3dc3c3-42d8-4d17-b244-c98f1f9560c7","Handler":null,"Name":"","Endpoint":""}
Sep 17 14:05:21.566841 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:21.566818    2582 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 17 14:05:21.566958 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:21.566856    2582 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 17 14:05:21.604240 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:21.604204    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gtdk" event={"ID":"22d71632-956e-4285-9cc2-928ec6492277","Type":"ContainerStarted","Data":"34fa6ed34a180e7571f7b14cbe80790447c22d093c3bac5e23cb06bb991e19de"}
Sep 17 14:05:21.607734 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:21.607702    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf" event={"ID":"7824d21a-4398-41f6-9303-501f38a2cfd4","Type":"ContainerStarted","Data":"518fbe4e4c00d79cb3da2e710f8e87af60483fa6ad418e845e62b527d86d98af"}
Sep 17 14:05:21.609280 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:21.609247    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jwwq2" event={"ID":"927c3577-20a9-4c54-8c98-0af3bcb37c06","Type":"ContainerStarted","Data":"9d9c24cbf00c425e5d0322fc0804f86c1c777e7456b1cd1e437f04087d59b50e"}
Sep 17 14:05:21.631894 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:21.631851    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-jwwq2" podStartSLOduration=7.048028948 podStartE2EDuration="21.631838022s" totalImagesPullingTime="14.583809074s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 14:05:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:05:21.631781825 +0000 UTC m=+22.633325300" watchObservedRunningTime="2026-09-17 14:05:21.631838022 +0000 UTC m=+22.633381496"
Sep 17 14:05:22.487428 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:22.487396    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dg4gz"
Sep 17 14:05:22.487622 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:22.487510    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dg4gz" podUID="a2cbe70b-eb96-4779-9143-b95a1204aae2"
Sep 17 14:05:23.487060 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:23.487026    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7k2wk"
Sep 17 14:05:23.487584 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:23.487073    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4p4g"
Sep 17 14:05:23.487584 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:23.487185    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7k2wk" podUID="31e72879-bbfc-4684-bcbb-fd79b4eec4b6"
Sep 17 14:05:23.487584 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:23.487278    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4p4g" podUID="059afb43-5e5e-4db9-bfa2-d874291c1c3a"
Sep 17 14:05:23.617749 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:23.617707    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gtdk" event={"ID":"22d71632-956e-4285-9cc2-928ec6492277","Type":"ContainerStarted","Data":"dc3dcab685140dbea222d47517b173f184011a6af0533b254e1c1ffbff213516"}
Sep 17 14:05:23.621646 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:23.621593    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf" event={"ID":"7824d21a-4398-41f6-9303-501f38a2cfd4","Type":"ContainerStarted","Data":"7739fd5a8b3feda82d4a821a4884a4c141a4e0c393d908b6766fddff861bd5ea"}
Sep 17 14:05:23.644316 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:23.644263    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gtdk" podStartSLOduration=3.038694074 podStartE2EDuration="23.644243488s" totalImagesPullingTime="20.605549414s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 14:05:00 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:05:23.643815174 +0000 UTC m=+24.645358671" watchObservedRunningTime="2026-09-17 14:05:23.644243488 +0000 UTC m=+24.645786966"
Sep 17 14:05:24.486808 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:24.486779    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dg4gz"
Sep 17 14:05:24.486956 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:24.486886    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dg4gz" podUID="a2cbe70b-eb96-4779-9143-b95a1204aae2"
Sep 17 14:05:24.972844 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:24.972809    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-2j77v"
Sep 17 14:05:24.973462 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:24.973440    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-2j77v"
Sep 17 14:05:25.487805 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:25.487627    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7k2wk"
Sep 17 14:05:25.487950 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:25.487679    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4p4g"
Sep 17 14:05:25.487950 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:25.487907    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7k2wk" podUID="31e72879-bbfc-4684-bcbb-fd79b4eec4b6"
Sep 17 14:05:25.488065 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:25.487964    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4p4g" podUID="059afb43-5e5e-4db9-bfa2-d874291c1c3a"
Sep 17 14:05:25.628722 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:25.628688    2582 generic.go:356] "Generic (PLEG): container finished" podID="82aa9496-50ec-4a83-9d0c-d227df61037a" containerID="f503d5b71222709a7dda95fd8e550e2f55f146156cc7b9b6f3f7ba5a52e170bb" exitCode=0
Sep 17 14:05:25.628885 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:25.628735    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xflkb" event={"ID":"82aa9496-50ec-4a83-9d0c-d227df61037a","Type":"ContainerDied","Data":"f503d5b71222709a7dda95fd8e550e2f55f146156cc7b9b6f3f7ba5a52e170bb"}
Sep 17 14:05:25.632260 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:25.632234    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf" event={"ID":"7824d21a-4398-41f6-9303-501f38a2cfd4","Type":"ContainerStarted","Data":"e8b51d53290d3dea5ce3eca4056ddc2763df11b40c991f9edf23284f173be10a"}
Sep 17 14:05:25.632569 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:25.632549    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-2j77v"
Sep 17 14:05:25.632974 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:25.632956    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-2j77v"
Sep 17 14:05:25.680241 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:25.678804    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf" podStartSLOduration=7.359164582 podStartE2EDuration="25.67878478s" totalImagesPullingTime="18.319620198s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 14:05:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:05:25.676495095 +0000 UTC m=+26.678038570" watchObservedRunningTime="2026-09-17 14:05:25.67878478 +0000 UTC m=+26.680328260"
Sep 17 14:05:26.487283 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:26.487253    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dg4gz"
Sep 17 14:05:26.487954 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:26.487357    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dg4gz" podUID="a2cbe70b-eb96-4779-9143-b95a1204aae2"
Sep 17 14:05:26.635912 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:26.635882    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:26.636077 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:26.635968    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:26.636077 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:26.636039    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:26.650430 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:26.650388    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:26.650673 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:26.650658    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:27.116795 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:27.116715    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7k2wk"]
Sep 17 14:05:27.117026 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:27.116823    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7k2wk"
Sep 17 14:05:27.117026 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:27.116902    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7k2wk" podUID="31e72879-bbfc-4684-bcbb-fd79b4eec4b6"
Sep 17 14:05:27.120719 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:27.120691    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-r4p4g"]
Sep 17 14:05:27.120845 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:27.120801    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4p4g"
Sep 17 14:05:27.120918 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:27.120902    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4p4g" podUID="059afb43-5e5e-4db9-bfa2-d874291c1c3a"
Sep 17 14:05:27.125405 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:27.125381    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dg4gz"]
Sep 17 14:05:27.125521 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:27.125462    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dg4gz"
Sep 17 14:05:27.125585 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:27.125566    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dg4gz" podUID="a2cbe70b-eb96-4779-9143-b95a1204aae2"
Sep 17 14:05:27.639698 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:27.639664    2582 generic.go:356] "Generic (PLEG): container finished" podID="82aa9496-50ec-4a83-9d0c-d227df61037a" containerID="7cd6100d97a2c3094c88b7f66063d2fc023d2ce35afa31a65f312ec52ac7af87" exitCode=0
Sep 17 14:05:27.640119 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:27.639755    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xflkb" event={"ID":"82aa9496-50ec-4a83-9d0c-d227df61037a","Type":"ContainerDied","Data":"7cd6100d97a2c3094c88b7f66063d2fc023d2ce35afa31a65f312ec52ac7af87"}
Sep 17 14:05:28.487016 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:28.486981    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4p4g"
Sep 17 14:05:28.487191 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:28.487020    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dg4gz"
Sep 17 14:05:28.487191 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:28.487025    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7k2wk"
Sep 17 14:05:28.487191 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:28.487093    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4p4g" podUID="059afb43-5e5e-4db9-bfa2-d874291c1c3a"
Sep 17 14:05:28.487191 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:28.487173    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dg4gz" podUID="a2cbe70b-eb96-4779-9143-b95a1204aae2"
Sep 17 14:05:28.487320 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:28.487243    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7k2wk" podUID="31e72879-bbfc-4684-bcbb-fd79b4eec4b6"
Sep 17 14:05:28.644240 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:28.644143    2582 generic.go:356] "Generic (PLEG): container finished" podID="82aa9496-50ec-4a83-9d0c-d227df61037a" containerID="e07a4a9c77cafb811024ab56bdd7bd7a8261d319d031188bf85d82b783cf182c" exitCode=0
Sep 17 14:05:28.644240 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:28.644225    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xflkb" event={"ID":"82aa9496-50ec-4a83-9d0c-d227df61037a","Type":"ContainerDied","Data":"e07a4a9c77cafb811024ab56bdd7bd7a8261d319d031188bf85d82b783cf182c"}
Sep 17 14:05:30.487510 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:30.487475    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dg4gz"
Sep 17 14:05:30.488030 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:30.487520    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7k2wk"
Sep 17 14:05:30.488030 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:30.487584    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4p4g"
Sep 17 14:05:30.488030 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:30.487598    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dg4gz" podUID="a2cbe70b-eb96-4779-9143-b95a1204aae2"
Sep 17 14:05:30.488030 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:30.487715    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4p4g" podUID="059afb43-5e5e-4db9-bfa2-d874291c1c3a"
Sep 17 14:05:30.488030 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:30.487801    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7k2wk" podUID="31e72879-bbfc-4684-bcbb-fd79b4eec4b6"
Sep 17 14:05:32.487285 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:32.487199    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7k2wk"
Sep 17 14:05:32.487285 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:32.487240    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dg4gz"
Sep 17 14:05:32.487968 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:32.487310    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4p4g"
Sep 17 14:05:32.487968 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:32.487330    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7k2wk" podUID="31e72879-bbfc-4684-bcbb-fd79b4eec4b6"
Sep 17 14:05:32.487968 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:32.487391    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r4p4g" podUID="059afb43-5e5e-4db9-bfa2-d874291c1c3a"
Sep 17 14:05:32.487968 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:32.487522    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dg4gz" podUID="a2cbe70b-eb96-4779-9143-b95a1204aae2"
Sep 17 14:05:32.782844 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:32.782814    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-53.ec2.internal" event="NodeReady"
Sep 17 14:05:32.783011 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:32.782954    2582 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 17 14:05:32.832564 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:32.832528    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-c7fc9df5d-jl62r"]
Sep 17 14:05:32.877449 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:32.877359    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-zjz6h"]
Sep 17 14:05:32.877618 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:32.877492    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c7fc9df5d-jl62r"
Sep 17 14:05:32.880020 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:32.879992    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 17 14:05:32.880742 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:32.880681    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-sjbd6\""
Sep 17 14:05:32.880742 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:32.880716    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 17 14:05:32.880947 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:32.880819    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 17 14:05:32.885422 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:32.885402    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 17 14:05:32.889990 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:32.889967    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-6v652"]
Sep 17 14:05:32.890102 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:32.890085    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zjz6h"
Sep 17 14:05:32.892665 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:32.892640    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 17 14:05:32.892779 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:32.892735    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 17 14:05:32.892917 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:32.892889    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-84cnx\""
Sep 17 14:05:32.892998 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:32.892889    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 17 14:05:32.910423 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:32.910392    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-c7fc9df5d-jl62r"]
Sep 17 14:05:32.910569 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:32.910441    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zjz6h"]
Sep 17 14:05:32.910569 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:32.910461    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6v652"]
Sep 17 14:05:32.910569 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:32.910528    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6v652"
Sep 17 14:05:32.912849 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:32.912826    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 17 14:05:32.912996 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:32.912834    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hcwc9\""
Sep 17 14:05:32.913111 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:32.913097    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 17 14:05:33.013051 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.013014    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bs8r\" (UniqueName: \"kubernetes.io/projected/81211573-af3a-4b68-ac05-92fdbb7cb8b2-kube-api-access-4bs8r\") pod \"image-registry-c7fc9df5d-jl62r\" (UID: \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\") " pod="openshift-image-registry/image-registry-c7fc9df5d-jl62r"
Sep 17 14:05:33.013219 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.013065    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5cd5a40b-4883-4928-8dd6-b0c3d13ad8ed-cert\") pod \"ingress-canary-zjz6h\" (UID: \"5cd5a40b-4883-4928-8dd6-b0c3d13ad8ed\") " pod="openshift-ingress-canary/ingress-canary-zjz6h"
Sep 17 14:05:33.013219 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.013091    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/40b77f59-aa77-4958-bd61-2522680f1ecf-config-volume\") pod \"dns-default-6v652\" (UID: \"40b77f59-aa77-4958-bd61-2522680f1ecf\") " pod="openshift-dns/dns-default-6v652"
Sep 17 14:05:33.013219 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.013105    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40b77f59-aa77-4958-bd61-2522680f1ecf-tmp-dir\") pod \"dns-default-6v652\" (UID: \"40b77f59-aa77-4958-bd61-2522680f1ecf\") " pod="openshift-dns/dns-default-6v652"
Sep 17 14:05:33.013219 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.013151    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5ts9\" (UniqueName: \"kubernetes.io/projected/5cd5a40b-4883-4928-8dd6-b0c3d13ad8ed-kube-api-access-w5ts9\") pod \"ingress-canary-zjz6h\" (UID: \"5cd5a40b-4883-4928-8dd6-b0c3d13ad8ed\") " pod="openshift-ingress-canary/ingress-canary-zjz6h"
Sep 17 14:05:33.013219 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.013182    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/81211573-af3a-4b68-ac05-92fdbb7cb8b2-registry-tls\") pod \"image-registry-c7fc9df5d-jl62r\" (UID: \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\") " pod="openshift-image-registry/image-registry-c7fc9df5d-jl62r"
Sep 17 14:05:33.013219 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.013200    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/81211573-af3a-4b68-ac05-92fdbb7cb8b2-bound-sa-token\") pod \"image-registry-c7fc9df5d-jl62r\" (UID: \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\") " pod="openshift-image-registry/image-registry-c7fc9df5d-jl62r"
Sep 17 14:05:33.013219 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.013217    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/40b77f59-aa77-4958-bd61-2522680f1ecf-metrics-tls\") pod \"dns-default-6v652\" (UID: \"40b77f59-aa77-4958-bd61-2522680f1ecf\") " pod="openshift-dns/dns-default-6v652"
Sep 17 14:05:33.013518 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.013247    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/81211573-af3a-4b68-ac05-92fdbb7cb8b2-ca-trust-extracted\") pod \"image-registry-c7fc9df5d-jl62r\" (UID: \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\") " pod="openshift-image-registry/image-registry-c7fc9df5d-jl62r"
Sep 17 14:05:33.013518 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.013269    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/81211573-af3a-4b68-ac05-92fdbb7cb8b2-registry-certificates\") pod \"image-registry-c7fc9df5d-jl62r\" (UID: \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\") " pod="openshift-image-registry/image-registry-c7fc9df5d-jl62r"
Sep 17 14:05:33.013518 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.013285    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/81211573-af3a-4b68-ac05-92fdbb7cb8b2-installation-pull-secrets\") pod \"image-registry-c7fc9df5d-jl62r\" (UID: \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\") " pod="openshift-image-registry/image-registry-c7fc9df5d-jl62r"
Sep 17 14:05:33.013518 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.013301    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljv4v\" (UniqueName: \"kubernetes.io/projected/40b77f59-aa77-4958-bd61-2522680f1ecf-kube-api-access-ljv4v\") pod \"dns-default-6v652\" (UID: \"40b77f59-aa77-4958-bd61-2522680f1ecf\") " pod="openshift-dns/dns-default-6v652"
Sep 17 14:05:33.013518 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.013378    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/81211573-af3a-4b68-ac05-92fdbb7cb8b2-image-registry-private-configuration\") pod \"image-registry-c7fc9df5d-jl62r\" (UID: \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\") " pod="openshift-image-registry/image-registry-c7fc9df5d-jl62r"
Sep 17 14:05:33.013518 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.013418    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/81211573-af3a-4b68-ac05-92fdbb7cb8b2-trusted-ca\") pod \"image-registry-c7fc9df5d-jl62r\" (UID: \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\") " pod="openshift-image-registry/image-registry-c7fc9df5d-jl62r"
Sep 17 14:05:33.114381 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.114343    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/059afb43-5e5e-4db9-bfa2-d874291c1c3a-metrics-certs\") pod \"network-metrics-daemon-r4p4g\" (UID: \"059afb43-5e5e-4db9-bfa2-d874291c1c3a\") " pod="openshift-multus/network-metrics-daemon-r4p4g"
Sep 17 14:05:33.114381 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.114381    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5cd5a40b-4883-4928-8dd6-b0c3d13ad8ed-cert\") pod \"ingress-canary-zjz6h\" (UID: \"5cd5a40b-4883-4928-8dd6-b0c3d13ad8ed\") " pod="openshift-ingress-canary/ingress-canary-zjz6h"
Sep 17 14:05:33.114646 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.114407    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/40b77f59-aa77-4958-bd61-2522680f1ecf-config-volume\") pod \"dns-default-6v652\" (UID: \"40b77f59-aa77-4958-bd61-2522680f1ecf\") " pod="openshift-dns/dns-default-6v652"
Sep 17 14:05:33.114646 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.114422    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40b77f59-aa77-4958-bd61-2522680f1ecf-tmp-dir\") pod \"dns-default-6v652\" (UID: \"40b77f59-aa77-4958-bd61-2522680f1ecf\") " pod="openshift-dns/dns-default-6v652"
Sep 17 14:05:33.114646 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.114441    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5ts9\" (UniqueName: \"kubernetes.io/projected/5cd5a40b-4883-4928-8dd6-b0c3d13ad8ed-kube-api-access-w5ts9\") pod \"ingress-canary-zjz6h\" (UID: \"5cd5a40b-4883-4928-8dd6-b0c3d13ad8ed\") " pod="openshift-ingress-canary/ingress-canary-zjz6h"
Sep 17 14:05:33.114646 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:33.114510    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 14:05:33.114646 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.114552    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/81211573-af3a-4b68-ac05-92fdbb7cb8b2-registry-tls\") pod \"image-registry-c7fc9df5d-jl62r\" (UID: \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\") " pod="openshift-image-registry/image-registry-c7fc9df5d-jl62r"
Sep 17 14:05:33.114646 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.114576    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/81211573-af3a-4b68-ac05-92fdbb7cb8b2-bound-sa-token\") pod \"image-registry-c7fc9df5d-jl62r\" (UID: \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\") " pod="openshift-image-registry/image-registry-c7fc9df5d-jl62r"
Sep 17 14:05:33.114646 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:33.114601    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/059afb43-5e5e-4db9-bfa2-d874291c1c3a-metrics-certs podName:059afb43-5e5e-4db9-bfa2-d874291c1c3a nodeName:}" failed. No retries permitted until 2026-09-17 14:06:05.114579239 +0000 UTC m=+66.116122691 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/059afb43-5e5e-4db9-bfa2-d874291c1c3a-metrics-certs") pod "network-metrics-daemon-r4p4g" (UID: "059afb43-5e5e-4db9-bfa2-d874291c1c3a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 14:05:33.114958 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.114650    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/40b77f59-aa77-4958-bd61-2522680f1ecf-metrics-tls\") pod \"dns-default-6v652\" (UID: \"40b77f59-aa77-4958-bd61-2522680f1ecf\") " pod="openshift-dns/dns-default-6v652"
Sep 17 14:05:33.114958 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.114685    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31e72879-bbfc-4684-bcbb-fd79b4eec4b6-original-pull-secret\") pod \"global-pull-secret-syncer-7k2wk\" (UID: \"31e72879-bbfc-4684-bcbb-fd79b4eec4b6\") " pod="kube-system/global-pull-secret-syncer-7k2wk"
Sep 17 14:05:33.114958 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.114712    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/81211573-af3a-4b68-ac05-92fdbb7cb8b2-ca-trust-extracted\") pod \"image-registry-c7fc9df5d-jl62r\" (UID: \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\") " pod="openshift-image-registry/image-registry-c7fc9df5d-jl62r"
Sep 17 14:05:33.114958 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.114735    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/81211573-af3a-4b68-ac05-92fdbb7cb8b2-registry-certificates\") pod \"image-registry-c7fc9df5d-jl62r\" (UID: \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\") " pod="openshift-image-registry/image-registry-c7fc9df5d-jl62r"
Sep 17 14:05:33.114958 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.114757    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/81211573-af3a-4b68-ac05-92fdbb7cb8b2-installation-pull-secrets\") pod \"image-registry-c7fc9df5d-jl62r\" (UID: \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\") " pod="openshift-image-registry/image-registry-c7fc9df5d-jl62r"
Sep 17 14:05:33.114958 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.114761    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40b77f59-aa77-4958-bd61-2522680f1ecf-tmp-dir\") pod \"dns-default-6v652\" (UID: \"40b77f59-aa77-4958-bd61-2522680f1ecf\") " pod="openshift-dns/dns-default-6v652"
Sep 17 14:05:33.114958 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.114779    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljv4v\" (UniqueName: \"kubernetes.io/projected/40b77f59-aa77-4958-bd61-2522680f1ecf-kube-api-access-ljv4v\") pod \"dns-default-6v652\" (UID: \"40b77f59-aa77-4958-bd61-2522680f1ecf\") " pod="openshift-dns/dns-default-6v652"
Sep 17 14:05:33.114958 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:33.114831    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 14:05:33.114958 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:33.114847    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c7fc9df5d-jl62r: secret "image-registry-tls" not found
Sep 17 14:05:33.114958 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:33.114892    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/81211573-af3a-4b68-ac05-92fdbb7cb8b2-registry-tls podName:81211573-af3a-4b68-ac05-92fdbb7cb8b2 nodeName:}" failed. No retries permitted until 2026-09-17 14:05:33.614875202 +0000 UTC m=+34.616418676 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/81211573-af3a-4b68-ac05-92fdbb7cb8b2-registry-tls") pod "image-registry-c7fc9df5d-jl62r" (UID: "81211573-af3a-4b68-ac05-92fdbb7cb8b2") : secret "image-registry-tls" not found
Sep 17 14:05:33.114958 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:33.114900    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 14:05:33.114958 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:33.114963    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40b77f59-aa77-4958-bd61-2522680f1ecf-metrics-tls podName:40b77f59-aa77-4958-bd61-2522680f1ecf nodeName:}" failed. No retries permitted until 2026-09-17 14:05:33.614946521 +0000 UTC m=+34.616489987 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/40b77f59-aa77-4958-bd61-2522680f1ecf-metrics-tls") pod "dns-default-6v652" (UID: "40b77f59-aa77-4958-bd61-2522680f1ecf") : secret "dns-default-metrics-tls" not found
Sep 17 14:05:33.115554 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:33.115027    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 14:05:33.115554 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.115055    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/40b77f59-aa77-4958-bd61-2522680f1ecf-config-volume\") pod \"dns-default-6v652\" (UID: \"40b77f59-aa77-4958-bd61-2522680f1ecf\") " pod="openshift-dns/dns-default-6v652"
Sep 17 14:05:33.115554 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:33.115061    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31e72879-bbfc-4684-bcbb-fd79b4eec4b6-original-pull-secret podName:31e72879-bbfc-4684-bcbb-fd79b4eec4b6 nodeName:}" failed. No retries permitted until 2026-09-17 14:06:05.115049587 +0000 UTC m=+66.116593042 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/31e72879-bbfc-4684-bcbb-fd79b4eec4b6-original-pull-secret") pod "global-pull-secret-syncer-7k2wk" (UID: "31e72879-bbfc-4684-bcbb-fd79b4eec4b6") : object "kube-system"/"original-pull-secret" not registered
Sep 17 14:05:33.115554 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.115119    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/81211573-af3a-4b68-ac05-92fdbb7cb8b2-ca-trust-extracted\") pod \"image-registry-c7fc9df5d-jl62r\" (UID: \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\") " pod="openshift-image-registry/image-registry-c7fc9df5d-jl62r"
Sep 17 14:05:33.115554 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.115143    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/81211573-af3a-4b68-ac05-92fdbb7cb8b2-image-registry-private-configuration\") pod \"image-registry-c7fc9df5d-jl62r\" (UID: \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\") " pod="openshift-image-registry/image-registry-c7fc9df5d-jl62r"
Sep 17 14:05:33.115554 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.115169    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/81211573-af3a-4b68-ac05-92fdbb7cb8b2-trusted-ca\") pod \"image-registry-c7fc9df5d-jl62r\" (UID: \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\") " pod="openshift-image-registry/image-registry-c7fc9df5d-jl62r"
Sep 17 14:05:33.115554 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.115195    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bs8r\" (UniqueName: \"kubernetes.io/projected/81211573-af3a-4b68-ac05-92fdbb7cb8b2-kube-api-access-4bs8r\") pod \"image-registry-c7fc9df5d-jl62r\" (UID: \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\") " pod="openshift-image-registry/image-registry-c7fc9df5d-jl62r"
Sep 17 14:05:33.115554 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.115423    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/81211573-af3a-4b68-ac05-92fdbb7cb8b2-registry-certificates\") pod \"image-registry-c7fc9df5d-jl62r\" (UID: \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\") " pod="openshift-image-registry/image-registry-c7fc9df5d-jl62r"
Sep 17 14:05:33.115912 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.115889    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/81211573-af3a-4b68-ac05-92fdbb7cb8b2-trusted-ca\") pod \"image-registry-c7fc9df5d-jl62r\" (UID: \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\") " pod="openshift-image-registry/image-registry-c7fc9df5d-jl62r"
Sep 17 14:05:33.118863 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.118758    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/81211573-af3a-4b68-ac05-92fdbb7cb8b2-installation-pull-secrets\") pod \"image-registry-c7fc9df5d-jl62r\" (UID: \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\") " pod="openshift-image-registry/image-registry-c7fc9df5d-jl62r"
Sep 17 14:05:33.118971 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.118759    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/81211573-af3a-4b68-ac05-92fdbb7cb8b2-image-registry-private-configuration\") pod \"image-registry-c7fc9df5d-jl62r\" (UID: \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\") " pod="openshift-image-registry/image-registry-c7fc9df5d-jl62r"
Sep 17 14:05:33.118971 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.118763    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5cd5a40b-4883-4928-8dd6-b0c3d13ad8ed-cert\") pod \"ingress-canary-zjz6h\" (UID: \"5cd5a40b-4883-4928-8dd6-b0c3d13ad8ed\") " pod="openshift-ingress-canary/ingress-canary-zjz6h"
Sep 17 14:05:33.123106 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.123078    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5ts9\" (UniqueName: \"kubernetes.io/projected/5cd5a40b-4883-4928-8dd6-b0c3d13ad8ed-kube-api-access-w5ts9\") pod \"ingress-canary-zjz6h\" (UID: \"5cd5a40b-4883-4928-8dd6-b0c3d13ad8ed\") " pod="openshift-ingress-canary/ingress-canary-zjz6h"
Sep 17 14:05:33.123658 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.123404    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/81211573-af3a-4b68-ac05-92fdbb7cb8b2-bound-sa-token\") pod \"image-registry-c7fc9df5d-jl62r\" (UID: \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\") " pod="openshift-image-registry/image-registry-c7fc9df5d-jl62r"
Sep 17 14:05:33.123658 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.123582    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bs8r\" (UniqueName: \"kubernetes.io/projected/81211573-af3a-4b68-ac05-92fdbb7cb8b2-kube-api-access-4bs8r\") pod \"image-registry-c7fc9df5d-jl62r\" (UID: \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\") " pod="openshift-image-registry/image-registry-c7fc9df5d-jl62r"
Sep 17 14:05:33.124037 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.124020    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljv4v\" (UniqueName: \"kubernetes.io/projected/40b77f59-aa77-4958-bd61-2522680f1ecf-kube-api-access-ljv4v\") pod \"dns-default-6v652\" (UID: \"40b77f59-aa77-4958-bd61-2522680f1ecf\") " pod="openshift-dns/dns-default-6v652"
Sep 17 14:05:33.200126 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.200024    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zjz6h"
Sep 17 14:05:33.216108 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.216069    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvl8x\" (UniqueName: \"kubernetes.io/projected/a2cbe70b-eb96-4779-9143-b95a1204aae2-kube-api-access-nvl8x\") pod \"network-check-target-dg4gz\" (UID: \"a2cbe70b-eb96-4779-9143-b95a1204aae2\") " pod="openshift-network-diagnostics/network-check-target-dg4gz"
Sep 17 14:05:33.219470 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:33.216523    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 14:05:33.219470 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:33.216560    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 14:05:33.219470 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:33.216585    2582 projected.go:196] Error preparing data for projected volume kube-api-access-nvl8x for pod openshift-network-diagnostics/network-check-target-dg4gz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 14:05:33.219470 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:33.216682    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a2cbe70b-eb96-4779-9143-b95a1204aae2-kube-api-access-nvl8x podName:a2cbe70b-eb96-4779-9143-b95a1204aae2 nodeName:}" failed. No retries permitted until 2026-09-17 14:06:05.216654311 +0000 UTC m=+66.218197779 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-nvl8x" (UniqueName: "kubernetes.io/projected/a2cbe70b-eb96-4779-9143-b95a1204aae2-kube-api-access-nvl8x") pod "network-check-target-dg4gz" (UID: "a2cbe70b-eb96-4779-9143-b95a1204aae2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 14:05:33.619699 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.619667    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/81211573-af3a-4b68-ac05-92fdbb7cb8b2-registry-tls\") pod \"image-registry-c7fc9df5d-jl62r\" (UID: \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\") " pod="openshift-image-registry/image-registry-c7fc9df5d-jl62r"
Sep 17 14:05:33.619699 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.619703    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/40b77f59-aa77-4958-bd61-2522680f1ecf-metrics-tls\") pod \"dns-default-6v652\" (UID: \"40b77f59-aa77-4958-bd61-2522680f1ecf\") " pod="openshift-dns/dns-default-6v652"
Sep 17 14:05:33.620424 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:33.619811    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 14:05:33.620424 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:33.619831    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c7fc9df5d-jl62r: secret "image-registry-tls" not found
Sep 17 14:05:33.620424 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:33.619854    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 14:05:33.620424 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:33.619885    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/81211573-af3a-4b68-ac05-92fdbb7cb8b2-registry-tls podName:81211573-af3a-4b68-ac05-92fdbb7cb8b2 nodeName:}" failed. No retries permitted until 2026-09-17 14:05:34.61987008 +0000 UTC m=+35.621413533 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/81211573-af3a-4b68-ac05-92fdbb7cb8b2-registry-tls") pod "image-registry-c7fc9df5d-jl62r" (UID: "81211573-af3a-4b68-ac05-92fdbb7cb8b2") : secret "image-registry-tls" not found
Sep 17 14:05:33.620424 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:33.619899    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40b77f59-aa77-4958-bd61-2522680f1ecf-metrics-tls podName:40b77f59-aa77-4958-bd61-2522680f1ecf nodeName:}" failed. No retries permitted until 2026-09-17 14:05:34.619893365 +0000 UTC m=+35.621436817 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/40b77f59-aa77-4958-bd61-2522680f1ecf-metrics-tls") pod "dns-default-6v652" (UID: "40b77f59-aa77-4958-bd61-2522680f1ecf") : secret "dns-default-metrics-tls" not found
Sep 17 14:05:33.653667 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.653640    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zjz6h"]
Sep 17 14:05:33.657366 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:05:33.657340    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cd5a40b_4883_4928_8dd6_b0c3d13ad8ed.slice/crio-b5468fb1ff79b22cc1400b8ecaaaba7a591e2e4df9b419bbaf11e0a657121ad5 WatchSource:0}: Error finding container b5468fb1ff79b22cc1400b8ecaaaba7a591e2e4df9b419bbaf11e0a657121ad5: Status 404 returned error can't find the container with id b5468fb1ff79b22cc1400b8ecaaaba7a591e2e4df9b419bbaf11e0a657121ad5
Sep 17 14:05:33.659196 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.659167    2582 generic.go:356] "Generic (PLEG): container finished" podID="82aa9496-50ec-4a83-9d0c-d227df61037a" containerID="23bdb7ac1e8494455915701e9c73b0bebc4eae720d2488550934afc7072ecad1" exitCode=0
Sep 17 14:05:33.659289 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:33.659245    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xflkb" event={"ID":"82aa9496-50ec-4a83-9d0c-d227df61037a","Type":"ContainerDied","Data":"23bdb7ac1e8494455915701e9c73b0bebc4eae720d2488550934afc7072ecad1"}
Sep 17 14:05:34.487205 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:34.487161    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7k2wk"
Sep 17 14:05:34.487386 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:34.487280    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4p4g"
Sep 17 14:05:34.487386 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:34.487319    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dg4gz"
Sep 17 14:05:34.489887 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:34.489865    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 17 14:05:34.491049 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:34.490863    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-27m2f\""
Sep 17 14:05:34.491049 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:34.490923    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 17 14:05:34.491049 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:34.490927    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 17 14:05:34.491049 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:34.490987    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lkjmx\""
Sep 17 14:05:34.491049 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:34.490863    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 17 14:05:34.629694 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:34.629656    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/81211573-af3a-4b68-ac05-92fdbb7cb8b2-registry-tls\") pod \"image-registry-c7fc9df5d-jl62r\" (UID: \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\") " pod="openshift-image-registry/image-registry-c7fc9df5d-jl62r"
Sep 17 14:05:34.629694 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:34.629699    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/40b77f59-aa77-4958-bd61-2522680f1ecf-metrics-tls\") pod \"dns-default-6v652\" (UID: \"40b77f59-aa77-4958-bd61-2522680f1ecf\") " pod="openshift-dns/dns-default-6v652"
Sep 17 14:05:34.630261 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:34.629822    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 14:05:34.630261 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:34.629826    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 14:05:34.630261 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:34.629850    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c7fc9df5d-jl62r: secret "image-registry-tls" not found
Sep 17 14:05:34.630261 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:34.629893    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40b77f59-aa77-4958-bd61-2522680f1ecf-metrics-tls podName:40b77f59-aa77-4958-bd61-2522680f1ecf nodeName:}" failed. No retries permitted until 2026-09-17 14:05:36.629871681 +0000 UTC m=+37.631415139 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/40b77f59-aa77-4958-bd61-2522680f1ecf-metrics-tls") pod "dns-default-6v652" (UID: "40b77f59-aa77-4958-bd61-2522680f1ecf") : secret "dns-default-metrics-tls" not found
Sep 17 14:05:34.630261 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:34.629912    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/81211573-af3a-4b68-ac05-92fdbb7cb8b2-registry-tls podName:81211573-af3a-4b68-ac05-92fdbb7cb8b2 nodeName:}" failed. No retries permitted until 2026-09-17 14:05:36.62990233 +0000 UTC m=+37.631445782 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/81211573-af3a-4b68-ac05-92fdbb7cb8b2-registry-tls") pod "image-registry-c7fc9df5d-jl62r" (UID: "81211573-af3a-4b68-ac05-92fdbb7cb8b2") : secret "image-registry-tls" not found
Sep 17 14:05:34.663681 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:34.663645    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zjz6h" event={"ID":"5cd5a40b-4883-4928-8dd6-b0c3d13ad8ed","Type":"ContainerStarted","Data":"b5468fb1ff79b22cc1400b8ecaaaba7a591e2e4df9b419bbaf11e0a657121ad5"}
Sep 17 14:05:34.667264 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:34.667233    2582 generic.go:356] "Generic (PLEG): container finished" podID="82aa9496-50ec-4a83-9d0c-d227df61037a" containerID="73368bbf6c26e583e7cb9f87949d5ffab73a40e720dce20b9e773939d693ce06" exitCode=0
Sep 17 14:05:34.667417 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:34.667300    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xflkb" event={"ID":"82aa9496-50ec-4a83-9d0c-d227df61037a","Type":"ContainerDied","Data":"73368bbf6c26e583e7cb9f87949d5ffab73a40e720dce20b9e773939d693ce06"}
Sep 17 14:05:35.674678 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.674415    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xflkb" event={"ID":"82aa9496-50ec-4a83-9d0c-d227df61037a","Type":"ContainerStarted","Data":"f5141d8c6bf88e314c64b6a2357110b7c76a428acb3cf475b6a7dbf112c6b9ba"}
Sep 17 14:05:35.698468 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.698408    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xflkb" podStartSLOduration=6.40285656 podStartE2EDuration="35.698386033s" totalImagesPullingTime="29.295529473s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 14:05:00 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:05:35.696921815 +0000 UTC m=+36.698465305" watchObservedRunningTime="2026-09-17 14:05:35.698386033 +0000 UTC m=+36.699929507"
Sep 17 14:05:35.770331 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.770296    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-68854cb984-hm4cb"]
Sep 17 14:05:35.794164 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.794134    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-68854cb984-hm4cb"]
Sep 17 14:05:35.794344 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.794264    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-68854cb984-hm4cb"
Sep 17 14:05:35.796455 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.796430    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 17 14:05:35.796843 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.796682    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 17 14:05:35.796843 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.796692    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 17 14:05:35.796843 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.796730    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 17 14:05:35.796843 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.796747    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 17 14:05:35.797119 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.796954    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 17 14:05:35.797119 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.796991    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-4r2cr\""
Sep 17 14:05:35.884483 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.884455    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-rr97k"]
Sep 17 14:05:35.890470 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.890440    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5844b856db-thrb2"]
Sep 17 14:05:35.890886 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.890850    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-rr97k"
Sep 17 14:05:35.892985 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.892963    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 17 14:05:35.894143 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.894095    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 17 14:05:35.894293 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.894262    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:35.894483 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.894465    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 17 14:05:35.894706 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.894659    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 17 14:05:35.895147 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.895038    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-8tbk4\""
Sep 17 14:05:35.896401 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.896383    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-r58pd"]
Sep 17 14:05:35.897094 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.896509    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-thrb2"
Sep 17 14:05:35.898888 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.898819    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 17 14:05:35.898888 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.898878    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 17 14:05:35.899069 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.898927    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 17 14:05:35.899069 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.898941    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 17 14:05:35.899226 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.899203    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 17 14:05:35.899323 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.899212    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-sktpt\""
Sep 17 14:05:35.903177 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.902940    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-rr97k"]
Sep 17 14:05:35.903177 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.903038    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-r58pd"
Sep 17 14:05:35.908601 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.907204    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-7tbnk\""
Sep 17 14:05:35.908601 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.907632    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 17 14:05:35.908601 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.908283    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 17 14:05:35.908601 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.908597    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 17 14:05:35.910763 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.909572    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-thrb2"]
Sep 17 14:05:35.911414 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.911380    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 17 14:05:35.911500 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.911418    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 17 14:05:35.922520 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.922499    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-r58pd"]
Sep 17 14:05:35.941218 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.941106    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2p2b\" (UniqueName: \"kubernetes.io/projected/1414b9c2-7c96-4383-9947-e7abcd68a074-kube-api-access-q2p2b\") pod \"router-default-68854cb984-hm4cb\" (UID: \"1414b9c2-7c96-4383-9947-e7abcd68a074\") " pod="openshift-ingress/router-default-68854cb984-hm4cb"
Sep 17 14:05:35.941218 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.941170    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/1414b9c2-7c96-4383-9947-e7abcd68a074-default-certificate\") pod \"router-default-68854cb984-hm4cb\" (UID: \"1414b9c2-7c96-4383-9947-e7abcd68a074\") " pod="openshift-ingress/router-default-68854cb984-hm4cb"
Sep 17 14:05:35.941218 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.941195    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1414b9c2-7c96-4383-9947-e7abcd68a074-metrics-certs\") pod \"router-default-68854cb984-hm4cb\" (UID: \"1414b9c2-7c96-4383-9947-e7abcd68a074\") " pod="openshift-ingress/router-default-68854cb984-hm4cb"
Sep 17 14:05:35.941457 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.941253    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1414b9c2-7c96-4383-9947-e7abcd68a074-service-ca-bundle\") pod \"router-default-68854cb984-hm4cb\" (UID: \"1414b9c2-7c96-4383-9947-e7abcd68a074\") " pod="openshift-ingress/router-default-68854cb984-hm4cb"
Sep 17 14:05:35.941457 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.941281    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/1414b9c2-7c96-4383-9947-e7abcd68a074-stats-auth\") pod \"router-default-68854cb984-hm4cb\" (UID: \"1414b9c2-7c96-4383-9947-e7abcd68a074\") " pod="openshift-ingress/router-default-68854cb984-hm4cb"
Sep 17 14:05:35.973374 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.973345    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-dhfb8"]
Sep 17 14:05:35.986075 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.986046    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wxbtg"]
Sep 17 14:05:35.986227 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.986179    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-dhfb8"
Sep 17 14:05:35.988572 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.988522    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 17 14:05:35.988717 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.988689    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 17 14:05:35.988796 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.988759    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-p8s96\""
Sep 17 14:05:35.996408 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.996318    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-47zgn"]
Sep 17 14:05:35.996533 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.996442    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wxbtg"
Sep 17 14:05:35.999742 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.999543    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 17 14:05:35.999742 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.999566    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 17 14:05:35.999742 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:35.999547    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-vgc4m\""
Sep 17 14:05:36.000264 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.000244    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 17 14:05:36.000579 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.000313    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 17 14:05:36.010768 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.010746    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-jl6xc"]
Sep 17 14:05:36.010889 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.010837    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-47zgn"
Sep 17 14:05:36.013446 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.013427    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 17 14:05:36.013577 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.013561    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 17 14:05:36.013657 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.013506    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 17 14:05:36.013657 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.013534    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-jbr7g\""
Sep 17 14:05:36.026146 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.026121    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-dhfb8"]
Sep 17 14:05:36.026254 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.026161    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-47zgn"]
Sep 17 14:05:36.026254 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.026189    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wxbtg"]
Sep 17 14:05:36.026254 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.026209    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-jl6xc"]
Sep 17 14:05:36.026370 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.026291    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jl6xc"
Sep 17 14:05:36.028834 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.028813    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 17 14:05:36.028834 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.028826    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 17 14:05:36.029004 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.028814    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 17 14:05:36.029004 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.028955    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-r2zkz\""
Sep 17 14:05:36.029109 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.029061    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 17 14:05:36.042266 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.042092    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1414b9c2-7c96-4383-9947-e7abcd68a074-service-ca-bundle\") pod \"router-default-68854cb984-hm4cb\" (UID: \"1414b9c2-7c96-4383-9947-e7abcd68a074\") " pod="openshift-ingress/router-default-68854cb984-hm4cb"
Sep 17 14:05:36.042266 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.042138    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/903d323b-ec37-4e01-89e9-7ab8caa061d7-trusted-ca-bundle\") pod \"insights-operator-5844b856db-thrb2\" (UID: \"903d323b-ec37-4e01-89e9-7ab8caa061d7\") " pod="openshift-insights/insights-operator-5844b856db-thrb2"
Sep 17 14:05:36.042266 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.042158    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/1414b9c2-7c96-4383-9947-e7abcd68a074-default-certificate\") pod \"router-default-68854cb984-hm4cb\" (UID: \"1414b9c2-7c96-4383-9947-e7abcd68a074\") " pod="openshift-ingress/router-default-68854cb984-hm4cb"
Sep 17 14:05:36.042266 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.042175    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/45ffa13d-635a-46e8-8894-30902b9cf719-serving-cert\") pod \"console-operator-696884fb9c-rr97k\" (UID: \"45ffa13d-635a-46e8-8894-30902b9cf719\") " pod="openshift-console-operator/console-operator-696884fb9c-rr97k"
Sep 17 14:05:36.042266 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.042198    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/45ffa13d-635a-46e8-8894-30902b9cf719-trusted-ca\") pod \"console-operator-696884fb9c-rr97k\" (UID: \"45ffa13d-635a-46e8-8894-30902b9cf719\") " pod="openshift-console-operator/console-operator-696884fb9c-rr97k"
Sep 17 14:05:36.042266 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.042235    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mbms\" (UniqueName: \"kubernetes.io/projected/903d323b-ec37-4e01-89e9-7ab8caa061d7-kube-api-access-4mbms\") pod \"insights-operator-5844b856db-thrb2\" (UID: \"903d323b-ec37-4e01-89e9-7ab8caa061d7\") " pod="openshift-insights/insights-operator-5844b856db-thrb2"
Sep 17 14:05:36.042648 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:36.042304    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1414b9c2-7c96-4383-9947-e7abcd68a074-service-ca-bundle podName:1414b9c2-7c96-4383-9947-e7abcd68a074 nodeName:}" failed. No retries permitted until 2026-09-17 14:05:36.542251414 +0000 UTC m=+37.543794875 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1414b9c2-7c96-4383-9947-e7abcd68a074-service-ca-bundle") pod "router-default-68854cb984-hm4cb" (UID: "1414b9c2-7c96-4383-9947-e7abcd68a074") : configmap references non-existent config key: service-ca.crt
Sep 17 14:05:36.042648 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.042385    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-r58pd\" (UID: \"1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-r58pd"
Sep 17 14:05:36.042648 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.042425    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wh828\" (UniqueName: \"kubernetes.io/projected/45ffa13d-635a-46e8-8894-30902b9cf719-kube-api-access-wh828\") pod \"console-operator-696884fb9c-rr97k\" (UID: \"45ffa13d-635a-46e8-8894-30902b9cf719\") " pod="openshift-console-operator/console-operator-696884fb9c-rr97k"
Sep 17 14:05:36.042648 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.042452    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-r58pd\" (UID: \"1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-r58pd"
Sep 17 14:05:36.042648 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.042492    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/903d323b-ec37-4e01-89e9-7ab8caa061d7-tmp\") pod \"insights-operator-5844b856db-thrb2\" (UID: \"903d323b-ec37-4e01-89e9-7ab8caa061d7\") " pod="openshift-insights/insights-operator-5844b856db-thrb2"
Sep 17 14:05:36.042957 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.042671    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/1414b9c2-7c96-4383-9947-e7abcd68a074-stats-auth\") pod \"router-default-68854cb984-hm4cb\" (UID: \"1414b9c2-7c96-4383-9947-e7abcd68a074\") " pod="openshift-ingress/router-default-68854cb984-hm4cb"
Sep 17 14:05:36.042957 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.042731    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1414b9c2-7c96-4383-9947-e7abcd68a074-metrics-certs\") pod \"router-default-68854cb984-hm4cb\" (UID: \"1414b9c2-7c96-4383-9947-e7abcd68a074\") " pod="openshift-ingress/router-default-68854cb984-hm4cb"
Sep 17 14:05:36.042957 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.042763    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2p2b\" (UniqueName: \"kubernetes.io/projected/1414b9c2-7c96-4383-9947-e7abcd68a074-kube-api-access-q2p2b\") pod \"router-default-68854cb984-hm4cb\" (UID: \"1414b9c2-7c96-4383-9947-e7abcd68a074\") " pod="openshift-ingress/router-default-68854cb984-hm4cb"
Sep 17 14:05:36.042957 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.042794    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/903d323b-ec37-4e01-89e9-7ab8caa061d7-snapshots\") pod \"insights-operator-5844b856db-thrb2\" (UID: \"903d323b-ec37-4e01-89e9-7ab8caa061d7\") " pod="openshift-insights/insights-operator-5844b856db-thrb2"
Sep 17 14:05:36.042957 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.042823    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/903d323b-ec37-4e01-89e9-7ab8caa061d7-service-ca-bundle\") pod \"insights-operator-5844b856db-thrb2\" (UID: \"903d323b-ec37-4e01-89e9-7ab8caa061d7\") " pod="openshift-insights/insights-operator-5844b856db-thrb2"
Sep 17 14:05:36.042957 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.042845    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/903d323b-ec37-4e01-89e9-7ab8caa061d7-serving-cert\") pod \"insights-operator-5844b856db-thrb2\" (UID: \"903d323b-ec37-4e01-89e9-7ab8caa061d7\") " pod="openshift-insights/insights-operator-5844b856db-thrb2"
Sep 17 14:05:36.042957 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.042872    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdv42\" (UniqueName: \"kubernetes.io/projected/1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99-kube-api-access-hdv42\") pod \"cluster-monitoring-operator-6cd954b897-r58pd\" (UID: \"1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-r58pd"
Sep 17 14:05:36.043287 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.043060    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/45ffa13d-635a-46e8-8894-30902b9cf719-config\") pod \"console-operator-696884fb9c-rr97k\" (UID: \"45ffa13d-635a-46e8-8894-30902b9cf719\") " pod="openshift-console-operator/console-operator-696884fb9c-rr97k"
Sep 17 14:05:36.043386 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:36.043363    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 17 14:05:36.043451 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:36.043430    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1414b9c2-7c96-4383-9947-e7abcd68a074-metrics-certs podName:1414b9c2-7c96-4383-9947-e7abcd68a074 nodeName:}" failed. No retries permitted until 2026-09-17 14:05:36.543417697 +0000 UTC m=+37.544961148 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1414b9c2-7c96-4383-9947-e7abcd68a074-metrics-certs") pod "router-default-68854cb984-hm4cb" (UID: "1414b9c2-7c96-4383-9947-e7abcd68a074") : secret "router-metrics-certs-default" not found
Sep 17 14:05:36.046547 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.046524    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/1414b9c2-7c96-4383-9947-e7abcd68a074-stats-auth\") pod \"router-default-68854cb984-hm4cb\" (UID: \"1414b9c2-7c96-4383-9947-e7abcd68a074\") " pod="openshift-ingress/router-default-68854cb984-hm4cb"
Sep 17 14:05:36.047012 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.046968    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/1414b9c2-7c96-4383-9947-e7abcd68a074-default-certificate\") pod \"router-default-68854cb984-hm4cb\" (UID: \"1414b9c2-7c96-4383-9947-e7abcd68a074\") " pod="openshift-ingress/router-default-68854cb984-hm4cb"
Sep 17 14:05:36.051861 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.051827    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2p2b\" (UniqueName: \"kubernetes.io/projected/1414b9c2-7c96-4383-9947-e7abcd68a074-kube-api-access-q2p2b\") pod \"router-default-68854cb984-hm4cb\" (UID: \"1414b9c2-7c96-4383-9947-e7abcd68a074\") " pod="openshift-ingress/router-default-68854cb984-hm4cb"
Sep 17 14:05:36.073819 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.073791    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-7hhfj"]
Sep 17 14:05:36.079304 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.079284    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-7hhfj"
Sep 17 14:05:36.082729 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.082705    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 17 14:05:36.082847 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.082739    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 17 14:05:36.084213 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.084189    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-9b46m\""
Sep 17 14:05:36.086582 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.086561    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-94gvr"]
Sep 17 14:05:36.093812 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.093788    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-7hhfj"]
Sep 17 14:05:36.093912 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.093894    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-94gvr"
Sep 17 14:05:36.096635 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.096600    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-k9nm6\""
Sep 17 14:05:36.107529 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.107504    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-94gvr"]
Sep 17 14:05:36.111620 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.111575    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:36.143438 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.143405    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wh828\" (UniqueName: \"kubernetes.io/projected/45ffa13d-635a-46e8-8894-30902b9cf719-kube-api-access-wh828\") pod \"console-operator-696884fb9c-rr97k\" (UID: \"45ffa13d-635a-46e8-8894-30902b9cf719\") " pod="openshift-console-operator/console-operator-696884fb9c-rr97k"
Sep 17 14:05:36.143438 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.143440    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-r58pd\" (UID: \"1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-r58pd"
Sep 17 14:05:36.143645 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.143460    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/903d323b-ec37-4e01-89e9-7ab8caa061d7-tmp\") pod \"insights-operator-5844b856db-thrb2\" (UID: \"903d323b-ec37-4e01-89e9-7ab8caa061d7\") " pod="openshift-insights/insights-operator-5844b856db-thrb2"
Sep 17 14:05:36.143645 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.143552    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkp2t\" (UniqueName: \"kubernetes.io/projected/a51d828f-334b-4be5-93f3-33eabaa37533-kube-api-access-vkp2t\") pod \"service-ca-operator-cf7c776b7-jl6xc\" (UID: \"a51d828f-334b-4be5-93f3-33eabaa37533\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jl6xc"
Sep 17 14:05:36.143645 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.143618    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/903d323b-ec37-4e01-89e9-7ab8caa061d7-snapshots\") pod \"insights-operator-5844b856db-thrb2\" (UID: \"903d323b-ec37-4e01-89e9-7ab8caa061d7\") " pod="openshift-insights/insights-operator-5844b856db-thrb2"
Sep 17 14:05:36.143645 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:36.143568    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 17 14:05:36.143783 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.143662    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a51d828f-334b-4be5-93f3-33eabaa37533-config\") pod \"service-ca-operator-cf7c776b7-jl6xc\" (UID: \"a51d828f-334b-4be5-93f3-33eabaa37533\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jl6xc"
Sep 17 14:05:36.143783 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.143693    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/903d323b-ec37-4e01-89e9-7ab8caa061d7-service-ca-bundle\") pod \"insights-operator-5844b856db-thrb2\" (UID: \"903d323b-ec37-4e01-89e9-7ab8caa061d7\") " pod="openshift-insights/insights-operator-5844b856db-thrb2"
Sep 17 14:05:36.143783 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:36.143715    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99-cluster-monitoring-operator-tls podName:1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99 nodeName:}" failed. No retries permitted until 2026-09-17 14:05:36.643690421 +0000 UTC m=+37.645233889 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-r58pd" (UID: "1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99") : secret "cluster-monitoring-operator-tls" not found
Sep 17 14:05:36.143783 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.143749    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/903d323b-ec37-4e01-89e9-7ab8caa061d7-serving-cert\") pod \"insights-operator-5844b856db-thrb2\" (UID: \"903d323b-ec37-4e01-89e9-7ab8caa061d7\") " pod="openshift-insights/insights-operator-5844b856db-thrb2"
Sep 17 14:05:36.143922 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.143812    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a51d828f-334b-4be5-93f3-33eabaa37533-serving-cert\") pod \"service-ca-operator-cf7c776b7-jl6xc\" (UID: \"a51d828f-334b-4be5-93f3-33eabaa37533\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jl6xc"
Sep 17 14:05:36.143922 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.143827    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/903d323b-ec37-4e01-89e9-7ab8caa061d7-tmp\") pod \"insights-operator-5844b856db-thrb2\" (UID: \"903d323b-ec37-4e01-89e9-7ab8caa061d7\") " pod="openshift-insights/insights-operator-5844b856db-thrb2"
Sep 17 14:05:36.143922 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.143844    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdv42\" (UniqueName: \"kubernetes.io/projected/1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99-kube-api-access-hdv42\") pod \"cluster-monitoring-operator-6cd954b897-r58pd\" (UID: \"1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-r58pd"
Sep 17 14:05:36.143922 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.143903    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/45ffa13d-635a-46e8-8894-30902b9cf719-config\") pod \"console-operator-696884fb9c-rr97k\" (UID: \"45ffa13d-635a-46e8-8894-30902b9cf719\") " pod="openshift-console-operator/console-operator-696884fb9c-rr97k"
Sep 17 14:05:36.144101 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.143927    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9857487e-aa03-45ac-88e4-e6a40b699fef-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-wxbtg\" (UID: \"9857487e-aa03-45ac-88e4-e6a40b699fef\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wxbtg"
Sep 17 14:05:36.144101 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.143975    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9fbd6f7b-2139-4134-9577-fce124537c73-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-47zgn\" (UID: \"9fbd6f7b-2139-4134-9577-fce124537c73\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-47zgn"
Sep 17 14:05:36.144101 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.144017    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/903d323b-ec37-4e01-89e9-7ab8caa061d7-trusted-ca-bundle\") pod \"insights-operator-5844b856db-thrb2\" (UID: \"903d323b-ec37-4e01-89e9-7ab8caa061d7\") " pod="openshift-insights/insights-operator-5844b856db-thrb2"
Sep 17 14:05:36.144101 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.144048    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nvhl\" (UniqueName: \"kubernetes.io/projected/9fbd6f7b-2139-4134-9577-fce124537c73-kube-api-access-9nvhl\") pod \"cluster-samples-operator-86499c89c4-47zgn\" (UID: \"9fbd6f7b-2139-4134-9577-fce124537c73\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-47zgn"
Sep 17 14:05:36.144101 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.144076    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/45ffa13d-635a-46e8-8894-30902b9cf719-serving-cert\") pod \"console-operator-696884fb9c-rr97k\" (UID: \"45ffa13d-635a-46e8-8894-30902b9cf719\") " pod="openshift-console-operator/console-operator-696884fb9c-rr97k"
Sep 17 14:05:36.144907 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.144108    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/45ffa13d-635a-46e8-8894-30902b9cf719-trusted-ca\") pod \"console-operator-696884fb9c-rr97k\" (UID: \"45ffa13d-635a-46e8-8894-30902b9cf719\") " pod="openshift-console-operator/console-operator-696884fb9c-rr97k"
Sep 17 14:05:36.144907 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.144162    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mbms\" (UniqueName: \"kubernetes.io/projected/903d323b-ec37-4e01-89e9-7ab8caa061d7-kube-api-access-4mbms\") pod \"insights-operator-5844b856db-thrb2\" (UID: \"903d323b-ec37-4e01-89e9-7ab8caa061d7\") " pod="openshift-insights/insights-operator-5844b856db-thrb2"
Sep 17 14:05:36.144907 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.144184    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9857487e-aa03-45ac-88e4-e6a40b699fef-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-wxbtg\" (UID: \"9857487e-aa03-45ac-88e4-e6a40b699fef\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wxbtg"
Sep 17 14:05:36.144907 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.144218    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nss7b\" (UniqueName: \"kubernetes.io/projected/9857487e-aa03-45ac-88e4-e6a40b699fef-kube-api-access-nss7b\") pod \"kube-storage-version-migrator-operator-6c8f84f897-wxbtg\" (UID: \"9857487e-aa03-45ac-88e4-e6a40b699fef\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wxbtg"
Sep 17 14:05:36.144907 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.144251    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-r58pd\" (UID: \"1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-r58pd"
Sep 17 14:05:36.144907 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.144274    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hplcv\" (UniqueName: \"kubernetes.io/projected/83ecb87d-85fa-4ac1-9800-103030f2d959-kube-api-access-hplcv\") pod \"volume-data-source-validator-86695b7cc7-dhfb8\" (UID: \"83ecb87d-85fa-4ac1-9800-103030f2d959\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-dhfb8"
Sep 17 14:05:36.144907 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.144337    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/903d323b-ec37-4e01-89e9-7ab8caa061d7-service-ca-bundle\") pod \"insights-operator-5844b856db-thrb2\" (UID: \"903d323b-ec37-4e01-89e9-7ab8caa061d7\") " pod="openshift-insights/insights-operator-5844b856db-thrb2"
Sep 17 14:05:36.144907 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.144567    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/903d323b-ec37-4e01-89e9-7ab8caa061d7-snapshots\") pod \"insights-operator-5844b856db-thrb2\" (UID: \"903d323b-ec37-4e01-89e9-7ab8caa061d7\") " pod="openshift-insights/insights-operator-5844b856db-thrb2"
Sep 17 14:05:36.144907 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.144602    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/45ffa13d-635a-46e8-8894-30902b9cf719-config\") pod \"console-operator-696884fb9c-rr97k\" (UID: \"45ffa13d-635a-46e8-8894-30902b9cf719\") " pod="openshift-console-operator/console-operator-696884fb9c-rr97k"
Sep 17 14:05:36.144907 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.144874    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/903d323b-ec37-4e01-89e9-7ab8caa061d7-trusted-ca-bundle\") pod \"insights-operator-5844b856db-thrb2\" (UID: \"903d323b-ec37-4e01-89e9-7ab8caa061d7\") " pod="openshift-insights/insights-operator-5844b856db-thrb2"
Sep 17 14:05:36.145244 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.145220    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/45ffa13d-635a-46e8-8894-30902b9cf719-trusted-ca\") pod \"console-operator-696884fb9c-rr97k\" (UID: \"45ffa13d-635a-46e8-8894-30902b9cf719\") " pod="openshift-console-operator/console-operator-696884fb9c-rr97k"
Sep 17 14:05:36.145292 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.145274    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-r58pd\" (UID: \"1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-r58pd"
Sep 17 14:05:36.146278 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.146256    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/903d323b-ec37-4e01-89e9-7ab8caa061d7-serving-cert\") pod \"insights-operator-5844b856db-thrb2\" (UID: \"903d323b-ec37-4e01-89e9-7ab8caa061d7\") " pod="openshift-insights/insights-operator-5844b856db-thrb2"
Sep 17 14:05:36.146522 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.146503    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/45ffa13d-635a-46e8-8894-30902b9cf719-serving-cert\") pod \"console-operator-696884fb9c-rr97k\" (UID: \"45ffa13d-635a-46e8-8894-30902b9cf719\") " pod="openshift-console-operator/console-operator-696884fb9c-rr97k"
Sep 17 14:05:36.153103 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.153080    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdv42\" (UniqueName: \"kubernetes.io/projected/1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99-kube-api-access-hdv42\") pod \"cluster-monitoring-operator-6cd954b897-r58pd\" (UID: \"1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-r58pd"
Sep 17 14:05:36.160750 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.160731    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wh828\" (UniqueName: \"kubernetes.io/projected/45ffa13d-635a-46e8-8894-30902b9cf719-kube-api-access-wh828\") pod \"console-operator-696884fb9c-rr97k\" (UID: \"45ffa13d-635a-46e8-8894-30902b9cf719\") " pod="openshift-console-operator/console-operator-696884fb9c-rr97k"
Sep 17 14:05:36.161140 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.161125    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mbms\" (UniqueName: \"kubernetes.io/projected/903d323b-ec37-4e01-89e9-7ab8caa061d7-kube-api-access-4mbms\") pod \"insights-operator-5844b856db-thrb2\" (UID: \"903d323b-ec37-4e01-89e9-7ab8caa061d7\") " pod="openshift-insights/insights-operator-5844b856db-thrb2"
Sep 17 14:05:36.206257 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.206227    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-rr97k"
Sep 17 14:05:36.211986 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.211959    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:36.214874 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.214858    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-thrb2"
Sep 17 14:05:36.245395 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.245373    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9857487e-aa03-45ac-88e4-e6a40b699fef-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-wxbtg\" (UID: \"9857487e-aa03-45ac-88e4-e6a40b699fef\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wxbtg"
Sep 17 14:05:36.245486 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.245407    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b9b37579-a215-4a77-bd0a-f33675b1b17f-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-7hhfj\" (UID: \"b9b37579-a215-4a77-bd0a-f33675b1b17f\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-7hhfj"
Sep 17 14:05:36.245486 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.245428    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b9b37579-a215-4a77-bd0a-f33675b1b17f-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-7hhfj\" (UID: \"b9b37579-a215-4a77-bd0a-f33675b1b17f\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-7hhfj"
Sep 17 14:05:36.245486 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.245455    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nss7b\" (UniqueName: \"kubernetes.io/projected/9857487e-aa03-45ac-88e4-e6a40b699fef-kube-api-access-nss7b\") pod \"kube-storage-version-migrator-operator-6c8f84f897-wxbtg\" (UID: \"9857487e-aa03-45ac-88e4-e6a40b699fef\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wxbtg"
Sep 17 14:05:36.245622 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.245529    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hplcv\" (UniqueName: \"kubernetes.io/projected/83ecb87d-85fa-4ac1-9800-103030f2d959-kube-api-access-hplcv\") pod \"volume-data-source-validator-86695b7cc7-dhfb8\" (UID: \"83ecb87d-85fa-4ac1-9800-103030f2d959\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-dhfb8"
Sep 17 14:05:36.245622 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.245575    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkwd8\" (UniqueName: \"kubernetes.io/projected/c93fff25-711a-4af0-90d8-08368c6161da-kube-api-access-jkwd8\") pod \"network-check-source-6f8b768455-94gvr\" (UID: \"c93fff25-711a-4af0-90d8-08368c6161da\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-94gvr"
Sep 17 14:05:36.245720 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.245645    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkp2t\" (UniqueName: \"kubernetes.io/projected/a51d828f-334b-4be5-93f3-33eabaa37533-kube-api-access-vkp2t\") pod \"service-ca-operator-cf7c776b7-jl6xc\" (UID: \"a51d828f-334b-4be5-93f3-33eabaa37533\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jl6xc"
Sep 17 14:05:36.245720 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.245690    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a51d828f-334b-4be5-93f3-33eabaa37533-config\") pod \"service-ca-operator-cf7c776b7-jl6xc\" (UID: \"a51d828f-334b-4be5-93f3-33eabaa37533\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jl6xc"
Sep 17 14:05:36.245811 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.245721    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a51d828f-334b-4be5-93f3-33eabaa37533-serving-cert\") pod \"service-ca-operator-cf7c776b7-jl6xc\" (UID: \"a51d828f-334b-4be5-93f3-33eabaa37533\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jl6xc"
Sep 17 14:05:36.245811 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.245781    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9857487e-aa03-45ac-88e4-e6a40b699fef-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-wxbtg\" (UID: \"9857487e-aa03-45ac-88e4-e6a40b699fef\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wxbtg"
Sep 17 14:05:36.245908 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.245828    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9fbd6f7b-2139-4134-9577-fce124537c73-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-47zgn\" (UID: \"9fbd6f7b-2139-4134-9577-fce124537c73\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-47zgn"
Sep 17 14:05:36.245908 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.245867    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nvhl\" (UniqueName: \"kubernetes.io/projected/9fbd6f7b-2139-4134-9577-fce124537c73-kube-api-access-9nvhl\") pod \"cluster-samples-operator-86499c89c4-47zgn\" (UID: \"9fbd6f7b-2139-4134-9577-fce124537c73\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-47zgn"
Sep 17 14:05:36.246007 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:36.245978    2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 17 14:05:36.246067 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.246003    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9857487e-aa03-45ac-88e4-e6a40b699fef-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-wxbtg\" (UID: \"9857487e-aa03-45ac-88e4-e6a40b699fef\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wxbtg"
Sep 17 14:05:36.246067 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:36.246062    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fbd6f7b-2139-4134-9577-fce124537c73-samples-operator-tls podName:9fbd6f7b-2139-4134-9577-fce124537c73 nodeName:}" failed. No retries permitted until 2026-09-17 14:05:36.74604457 +0000 UTC m=+37.747588035 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9fbd6f7b-2139-4134-9577-fce124537c73-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-47zgn" (UID: "9fbd6f7b-2139-4134-9577-fce124537c73") : secret "samples-operator-tls" not found
Sep 17 14:05:36.246348 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.246327    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a51d828f-334b-4be5-93f3-33eabaa37533-config\") pod \"service-ca-operator-cf7c776b7-jl6xc\" (UID: \"a51d828f-334b-4be5-93f3-33eabaa37533\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jl6xc"
Sep 17 14:05:36.248204 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.248183    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9857487e-aa03-45ac-88e4-e6a40b699fef-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-wxbtg\" (UID: \"9857487e-aa03-45ac-88e4-e6a40b699fef\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wxbtg"
Sep 17 14:05:36.248402 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.248382    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a51d828f-334b-4be5-93f3-33eabaa37533-serving-cert\") pod \"service-ca-operator-cf7c776b7-jl6xc\" (UID: \"a51d828f-334b-4be5-93f3-33eabaa37533\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jl6xc"
Sep 17 14:05:36.254857 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.254825    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkp2t\" (UniqueName: \"kubernetes.io/projected/a51d828f-334b-4be5-93f3-33eabaa37533-kube-api-access-vkp2t\") pod \"service-ca-operator-cf7c776b7-jl6xc\" (UID: \"a51d828f-334b-4be5-93f3-33eabaa37533\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jl6xc"
Sep 17 14:05:36.254951 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.254887    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hplcv\" (UniqueName: \"kubernetes.io/projected/83ecb87d-85fa-4ac1-9800-103030f2d959-kube-api-access-hplcv\") pod \"volume-data-source-validator-86695b7cc7-dhfb8\" (UID: \"83ecb87d-85fa-4ac1-9800-103030f2d959\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-dhfb8"
Sep 17 14:05:36.255516 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.255499    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nvhl\" (UniqueName: \"kubernetes.io/projected/9fbd6f7b-2139-4134-9577-fce124537c73-kube-api-access-9nvhl\") pod \"cluster-samples-operator-86499c89c4-47zgn\" (UID: \"9fbd6f7b-2139-4134-9577-fce124537c73\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-47zgn"
Sep 17 14:05:36.256048 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.256033    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nss7b\" (UniqueName: \"kubernetes.io/projected/9857487e-aa03-45ac-88e4-e6a40b699fef-kube-api-access-nss7b\") pod \"kube-storage-version-migrator-operator-6c8f84f897-wxbtg\" (UID: \"9857487e-aa03-45ac-88e4-e6a40b699fef\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wxbtg"
Sep 17 14:05:36.296883 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.296848    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-dhfb8"
Sep 17 14:05:36.308290 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.308264    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wxbtg"
Sep 17 14:05:36.320978 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.320940    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:36.335906 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.335879    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jl6xc"
Sep 17 14:05:36.347327 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.346946    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b9b37579-a215-4a77-bd0a-f33675b1b17f-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-7hhfj\" (UID: \"b9b37579-a215-4a77-bd0a-f33675b1b17f\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-7hhfj"
Sep 17 14:05:36.347327 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.346986    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b9b37579-a215-4a77-bd0a-f33675b1b17f-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-7hhfj\" (UID: \"b9b37579-a215-4a77-bd0a-f33675b1b17f\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-7hhfj"
Sep 17 14:05:36.347327 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.347041    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkwd8\" (UniqueName: \"kubernetes.io/projected/c93fff25-711a-4af0-90d8-08368c6161da-kube-api-access-jkwd8\") pod \"network-check-source-6f8b768455-94gvr\" (UID: \"c93fff25-711a-4af0-90d8-08368c6161da\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-94gvr"
Sep 17 14:05:36.347539 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:36.347444    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 17 14:05:36.347539 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:36.347538    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9b37579-a215-4a77-bd0a-f33675b1b17f-networking-console-plugin-cert podName:b9b37579-a215-4a77-bd0a-f33675b1b17f nodeName:}" failed. No retries permitted until 2026-09-17 14:05:36.847512171 +0000 UTC m=+37.849055638 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b9b37579-a215-4a77-bd0a-f33675b1b17f-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-7hhfj" (UID: "b9b37579-a215-4a77-bd0a-f33675b1b17f") : secret "networking-console-plugin-cert" not found
Sep 17 14:05:36.348060 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.348038    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b9b37579-a215-4a77-bd0a-f33675b1b17f-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-7hhfj\" (UID: \"b9b37579-a215-4a77-bd0a-f33675b1b17f\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-7hhfj"
Sep 17 14:05:36.357622 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.357578    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkwd8\" (UniqueName: \"kubernetes.io/projected/c93fff25-711a-4af0-90d8-08368c6161da-kube-api-access-jkwd8\") pod \"network-check-source-6f8b768455-94gvr\" (UID: \"c93fff25-711a-4af0-90d8-08368c6161da\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-94gvr"
Sep 17 14:05:36.405728 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.405696    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-94gvr"
Sep 17 14:05:36.422623 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.422585    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:36.491045 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.490976    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-rr97k"]
Sep 17 14:05:36.495589 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:05:36.495555    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45ffa13d_635a_46e8_8894_30902b9cf719.slice/crio-8d5d5fee2c80d74947308fe9458d31620d84ba8c80815fc9748a3ed7dec0da1b WatchSource:0}: Error finding container 8d5d5fee2c80d74947308fe9458d31620d84ba8c80815fc9748a3ed7dec0da1b: Status 404 returned error can't find the container with id 8d5d5fee2c80d74947308fe9458d31620d84ba8c80815fc9748a3ed7dec0da1b
Sep 17 14:05:36.513154 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.513046    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-thrb2"]
Sep 17 14:05:36.549562 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.548889    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1414b9c2-7c96-4383-9947-e7abcd68a074-metrics-certs\") pod \"router-default-68854cb984-hm4cb\" (UID: \"1414b9c2-7c96-4383-9947-e7abcd68a074\") " pod="openshift-ingress/router-default-68854cb984-hm4cb"
Sep 17 14:05:36.549562 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:36.549162    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 17 14:05:36.549562 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.549173    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1414b9c2-7c96-4383-9947-e7abcd68a074-service-ca-bundle\") pod \"router-default-68854cb984-hm4cb\" (UID: \"1414b9c2-7c96-4383-9947-e7abcd68a074\") " pod="openshift-ingress/router-default-68854cb984-hm4cb"
Sep 17 14:05:36.549562 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:36.549284    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1414b9c2-7c96-4383-9947-e7abcd68a074-metrics-certs podName:1414b9c2-7c96-4383-9947-e7abcd68a074 nodeName:}" failed. No retries permitted until 2026-09-17 14:05:37.549239995 +0000 UTC m=+38.550783470 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1414b9c2-7c96-4383-9947-e7abcd68a074-metrics-certs") pod "router-default-68854cb984-hm4cb" (UID: "1414b9c2-7c96-4383-9947-e7abcd68a074") : secret "router-metrics-certs-default" not found
Sep 17 14:05:36.549562 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:36.549361    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1414b9c2-7c96-4383-9947-e7abcd68a074-service-ca-bundle podName:1414b9c2-7c96-4383-9947-e7abcd68a074 nodeName:}" failed. No retries permitted until 2026-09-17 14:05:37.549344341 +0000 UTC m=+38.550887815 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1414b9c2-7c96-4383-9947-e7abcd68a074-service-ca-bundle") pod "router-default-68854cb984-hm4cb" (UID: "1414b9c2-7c96-4383-9947-e7abcd68a074") : configmap references non-existent config key: service-ca.crt
Sep 17 14:05:36.580844 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.580785    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-dhfb8"]
Sep 17 14:05:36.583101 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:05:36.583076    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83ecb87d_85fa_4ac1_9800_103030f2d959.slice/crio-b088d4150b2a63a4bf0c8bfd777f8a38c0312207233e5b7b415a553a4a035456 WatchSource:0}: Error finding container b088d4150b2a63a4bf0c8bfd777f8a38c0312207233e5b7b415a553a4a035456: Status 404 returned error can't find the container with id b088d4150b2a63a4bf0c8bfd777f8a38c0312207233e5b7b415a553a4a035456
Sep 17 14:05:36.651469 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.650746    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/81211573-af3a-4b68-ac05-92fdbb7cb8b2-registry-tls\") pod \"image-registry-c7fc9df5d-jl62r\" (UID: \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\") " pod="openshift-image-registry/image-registry-c7fc9df5d-jl62r"
Sep 17 14:05:36.651469 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.650791    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/40b77f59-aa77-4958-bd61-2522680f1ecf-metrics-tls\") pod \"dns-default-6v652\" (UID: \"40b77f59-aa77-4958-bd61-2522680f1ecf\") " pod="openshift-dns/dns-default-6v652"
Sep 17 14:05:36.651469 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.650853    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-r58pd\" (UID: \"1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-r58pd"
Sep 17 14:05:36.651469 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:36.651011    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 17 14:05:36.651469 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:36.651070    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99-cluster-monitoring-operator-tls podName:1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99 nodeName:}" failed. No retries permitted until 2026-09-17 14:05:37.651052259 +0000 UTC m=+38.652595724 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-r58pd" (UID: "1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99") : secret "cluster-monitoring-operator-tls" not found
Sep 17 14:05:36.651893 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:36.651627    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 14:05:36.651893 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:36.651700    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40b77f59-aa77-4958-bd61-2522680f1ecf-metrics-tls podName:40b77f59-aa77-4958-bd61-2522680f1ecf nodeName:}" failed. No retries permitted until 2026-09-17 14:05:40.65167855 +0000 UTC m=+41.653222022 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/40b77f59-aa77-4958-bd61-2522680f1ecf-metrics-tls") pod "dns-default-6v652" (UID: "40b77f59-aa77-4958-bd61-2522680f1ecf") : secret "dns-default-metrics-tls" not found
Sep 17 14:05:36.652122 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:36.652026    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 14:05:36.652122 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:36.652055    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c7fc9df5d-jl62r: secret "image-registry-tls" not found
Sep 17 14:05:36.652122 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:36.652104    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/81211573-af3a-4b68-ac05-92fdbb7cb8b2-registry-tls podName:81211573-af3a-4b68-ac05-92fdbb7cb8b2 nodeName:}" failed. No retries permitted until 2026-09-17 14:05:40.652088633 +0000 UTC m=+41.653632088 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/81211573-af3a-4b68-ac05-92fdbb7cb8b2-registry-tls") pod "image-registry-c7fc9df5d-jl62r" (UID: "81211573-af3a-4b68-ac05-92fdbb7cb8b2") : secret "image-registry-tls" not found
Sep 17 14:05:36.680524 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.680489    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zjz6h" event={"ID":"5cd5a40b-4883-4928-8dd6-b0c3d13ad8ed","Type":"ContainerStarted","Data":"8cf6674dc16a86a2a2837dedee058466ddd36618c378117a3cc095079b77fd00"}
Sep 17 14:05:36.681976 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.681944    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-rr97k" event={"ID":"45ffa13d-635a-46e8-8894-30902b9cf719","Type":"ContainerStarted","Data":"8d5d5fee2c80d74947308fe9458d31620d84ba8c80815fc9748a3ed7dec0da1b"}
Sep 17 14:05:36.683100 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.683081    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-thrb2" event={"ID":"903d323b-ec37-4e01-89e9-7ab8caa061d7","Type":"ContainerStarted","Data":"48fda95efe5f19fb53076c5919398140424f2f3ac291bb315fdfc005edcc24d6"}
Sep 17 14:05:36.684271 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.684250    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-dhfb8" event={"ID":"83ecb87d-85fa-4ac1-9800-103030f2d959","Type":"ContainerStarted","Data":"b088d4150b2a63a4bf0c8bfd777f8a38c0312207233e5b7b415a553a4a035456"}
Sep 17 14:05:36.700009 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.699964    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-zjz6h" podStartSLOduration=1.872461508 podStartE2EDuration="4.699948977s" totalImagesPullingTime="2.827487469s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:05:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:05:36.698554727 +0000 UTC m=+37.700098222" watchObservedRunningTime="2026-09-17 14:05:36.699948977 +0000 UTC m=+37.701492451"
Sep 17 14:05:36.751634 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.751583    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9fbd6f7b-2139-4134-9577-fce124537c73-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-47zgn\" (UID: \"9fbd6f7b-2139-4134-9577-fce124537c73\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-47zgn"
Sep 17 14:05:36.751789 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:36.751733    2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 17 14:05:36.751856 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:36.751800    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fbd6f7b-2139-4134-9577-fce124537c73-samples-operator-tls podName:9fbd6f7b-2139-4134-9577-fce124537c73 nodeName:}" failed. No retries permitted until 2026-09-17 14:05:37.751784741 +0000 UTC m=+38.753328209 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9fbd6f7b-2139-4134-9577-fce124537c73-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-47zgn" (UID: "9fbd6f7b-2139-4134-9577-fce124537c73") : secret "samples-operator-tls" not found
Sep 17 14:05:36.768740 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.768704    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-94gvr"]
Sep 17 14:05:36.771815 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.771752    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-jl6xc"]
Sep 17 14:05:36.772964 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.772943    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wxbtg"]
Sep 17 14:05:36.774374 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:05:36.774312    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc93fff25_711a_4af0_90d8_08368c6161da.slice/crio-3b304ffdde563794ec36de2c96e825cd84729b7ea51a987a229ac59e5e4de96d WatchSource:0}: Error finding container 3b304ffdde563794ec36de2c96e825cd84729b7ea51a987a229ac59e5e4de96d: Status 404 returned error can't find the container with id 3b304ffdde563794ec36de2c96e825cd84729b7ea51a987a229ac59e5e4de96d
Sep 17 14:05:36.775798 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:05:36.775700    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda51d828f_334b_4be5_93f3_33eabaa37533.slice/crio-d0f744ca35e6c9e214d2527842a5759c6c3bfa5530482249b84b59025f44d912 WatchSource:0}: Error finding container d0f744ca35e6c9e214d2527842a5759c6c3bfa5530482249b84b59025f44d912: Status 404 returned error can't find the container with id d0f744ca35e6c9e214d2527842a5759c6c3bfa5530482249b84b59025f44d912
Sep 17 14:05:36.776294 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:05:36.776275    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9857487e_aa03_45ac_88e4_e6a40b699fef.slice/crio-fea83f1f4eacf3f4bde04f13296106b9e9a5bd221705d0d0c36acaba9aeceb00 WatchSource:0}: Error finding container fea83f1f4eacf3f4bde04f13296106b9e9a5bd221705d0d0c36acaba9aeceb00: Status 404 returned error can't find the container with id fea83f1f4eacf3f4bde04f13296106b9e9a5bd221705d0d0c36acaba9aeceb00
Sep 17 14:05:36.790494 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.790467    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:36.852506 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:36.852471    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b9b37579-a215-4a77-bd0a-f33675b1b17f-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-7hhfj\" (UID: \"b9b37579-a215-4a77-bd0a-f33675b1b17f\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-7hhfj"
Sep 17 14:05:36.852665 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:36.852648    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 17 14:05:36.852722 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:36.852709    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9b37579-a215-4a77-bd0a-f33675b1b17f-networking-console-plugin-cert podName:b9b37579-a215-4a77-bd0a-f33675b1b17f nodeName:}" failed. No retries permitted until 2026-09-17 14:05:37.852692607 +0000 UTC m=+38.854236077 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b9b37579-a215-4a77-bd0a-f33675b1b17f-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-7hhfj" (UID: "b9b37579-a215-4a77-bd0a-f33675b1b17f") : secret "networking-console-plugin-cert" not found
Sep 17 14:05:37.205755 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:37.205717    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:37.403767 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:37.403688    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:37.561643 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:37.558419    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1414b9c2-7c96-4383-9947-e7abcd68a074-service-ca-bundle\") pod \"router-default-68854cb984-hm4cb\" (UID: \"1414b9c2-7c96-4383-9947-e7abcd68a074\") " pod="openshift-ingress/router-default-68854cb984-hm4cb"
Sep 17 14:05:37.561643 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:37.558582    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1414b9c2-7c96-4383-9947-e7abcd68a074-metrics-certs\") pod \"router-default-68854cb984-hm4cb\" (UID: \"1414b9c2-7c96-4383-9947-e7abcd68a074\") " pod="openshift-ingress/router-default-68854cb984-hm4cb"
Sep 17 14:05:37.561643 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:37.558742    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 17 14:05:37.561643 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:37.558807    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1414b9c2-7c96-4383-9947-e7abcd68a074-metrics-certs podName:1414b9c2-7c96-4383-9947-e7abcd68a074 nodeName:}" failed. No retries permitted until 2026-09-17 14:05:39.55878655 +0000 UTC m=+40.560330016 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1414b9c2-7c96-4383-9947-e7abcd68a074-metrics-certs") pod "router-default-68854cb984-hm4cb" (UID: "1414b9c2-7c96-4383-9947-e7abcd68a074") : secret "router-metrics-certs-default" not found
Sep 17 14:05:37.561643 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:37.559340    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1414b9c2-7c96-4383-9947-e7abcd68a074-service-ca-bundle podName:1414b9c2-7c96-4383-9947-e7abcd68a074 nodeName:}" failed. No retries permitted until 2026-09-17 14:05:39.559322887 +0000 UTC m=+40.560866354 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1414b9c2-7c96-4383-9947-e7abcd68a074-service-ca-bundle") pod "router-default-68854cb984-hm4cb" (UID: "1414b9c2-7c96-4383-9947-e7abcd68a074") : configmap references non-existent config key: service-ca.crt
Sep 17 14:05:37.660652 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:37.659867    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-r58pd\" (UID: \"1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-r58pd"
Sep 17 14:05:37.660652 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:37.660176    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 17 14:05:37.660652 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:37.660250    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99-cluster-monitoring-operator-tls podName:1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99 nodeName:}" failed. No retries permitted until 2026-09-17 14:05:39.660228455 +0000 UTC m=+40.661771934 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-r58pd" (UID: "1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99") : secret "cluster-monitoring-operator-tls" not found
Sep 17 14:05:37.690962 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:37.690882    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-94gvr" event={"ID":"c93fff25-711a-4af0-90d8-08368c6161da","Type":"ContainerStarted","Data":"3b304ffdde563794ec36de2c96e825cd84729b7ea51a987a229ac59e5e4de96d"}
Sep 17 14:05:37.693358 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:37.693285    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wxbtg" event={"ID":"9857487e-aa03-45ac-88e4-e6a40b699fef","Type":"ContainerStarted","Data":"fea83f1f4eacf3f4bde04f13296106b9e9a5bd221705d0d0c36acaba9aeceb00"}
Sep 17 14:05:37.698860 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:37.698799    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jl6xc" event={"ID":"a51d828f-334b-4be5-93f3-33eabaa37533","Type":"ContainerStarted","Data":"d0f744ca35e6c9e214d2527842a5759c6c3bfa5530482249b84b59025f44d912"}
Sep 17 14:05:37.761170 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:37.760941    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9fbd6f7b-2139-4134-9577-fce124537c73-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-47zgn\" (UID: \"9fbd6f7b-2139-4134-9577-fce124537c73\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-47zgn"
Sep 17 14:05:37.761170 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:37.761112    2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 17 14:05:37.761416 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:37.761188    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fbd6f7b-2139-4134-9577-fce124537c73-samples-operator-tls podName:9fbd6f7b-2139-4134-9577-fce124537c73 nodeName:}" failed. No retries permitted until 2026-09-17 14:05:39.761165969 +0000 UTC m=+40.762709424 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9fbd6f7b-2139-4134-9577-fce124537c73-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-47zgn" (UID: "9fbd6f7b-2139-4134-9577-fce124537c73") : secret "samples-operator-tls" not found
Sep 17 14:05:37.862738 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:37.862156    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b9b37579-a215-4a77-bd0a-f33675b1b17f-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-7hhfj\" (UID: \"b9b37579-a215-4a77-bd0a-f33675b1b17f\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-7hhfj"
Sep 17 14:05:37.862738 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:37.862567    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 17 14:05:37.862738 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:37.862668    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9b37579-a215-4a77-bd0a-f33675b1b17f-networking-console-plugin-cert podName:b9b37579-a215-4a77-bd0a-f33675b1b17f nodeName:}" failed. No retries permitted until 2026-09-17 14:05:39.862647624 +0000 UTC m=+40.864191091 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b9b37579-a215-4a77-bd0a-f33675b1b17f-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-7hhfj" (UID: "b9b37579-a215-4a77-bd0a-f33675b1b17f") : secret "networking-console-plugin-cert" not found
Sep 17 14:05:38.388164 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:38.388093    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:38.530186 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:38.530150    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:39.578777 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:39.578743    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1414b9c2-7c96-4383-9947-e7abcd68a074-metrics-certs\") pod \"router-default-68854cb984-hm4cb\" (UID: \"1414b9c2-7c96-4383-9947-e7abcd68a074\") " pod="openshift-ingress/router-default-68854cb984-hm4cb"
Sep 17 14:05:39.579253 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:39.578830    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1414b9c2-7c96-4383-9947-e7abcd68a074-service-ca-bundle\") pod \"router-default-68854cb984-hm4cb\" (UID: \"1414b9c2-7c96-4383-9947-e7abcd68a074\") " pod="openshift-ingress/router-default-68854cb984-hm4cb"
Sep 17 14:05:39.579253 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:39.579007    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1414b9c2-7c96-4383-9947-e7abcd68a074-service-ca-bundle podName:1414b9c2-7c96-4383-9947-e7abcd68a074 nodeName:}" failed. No retries permitted until 2026-09-17 14:05:43.578988886 +0000 UTC m=+44.580532354 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1414b9c2-7c96-4383-9947-e7abcd68a074-service-ca-bundle") pod "router-default-68854cb984-hm4cb" (UID: "1414b9c2-7c96-4383-9947-e7abcd68a074") : configmap references non-existent config key: service-ca.crt
Sep 17 14:05:39.579253 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:39.579121    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 17 14:05:39.579253 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:39.579191    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1414b9c2-7c96-4383-9947-e7abcd68a074-metrics-certs podName:1414b9c2-7c96-4383-9947-e7abcd68a074 nodeName:}" failed. No retries permitted until 2026-09-17 14:05:43.579174091 +0000 UTC m=+44.580717570 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1414b9c2-7c96-4383-9947-e7abcd68a074-metrics-certs") pod "router-default-68854cb984-hm4cb" (UID: "1414b9c2-7c96-4383-9947-e7abcd68a074") : secret "router-metrics-certs-default" not found
Sep 17 14:05:39.679676 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:39.679630    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-r58pd\" (UID: \"1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-r58pd"
Sep 17 14:05:39.679907 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:39.679775    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 17 14:05:39.679907 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:39.679845    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99-cluster-monitoring-operator-tls podName:1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99 nodeName:}" failed. No retries permitted until 2026-09-17 14:05:43.679827818 +0000 UTC m=+44.681371270 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-r58pd" (UID: "1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99") : secret "cluster-monitoring-operator-tls" not found
Sep 17 14:05:39.780317 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:39.780277    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9fbd6f7b-2139-4134-9577-fce124537c73-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-47zgn\" (UID: \"9fbd6f7b-2139-4134-9577-fce124537c73\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-47zgn"
Sep 17 14:05:39.780466 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:39.780443    2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 17 14:05:39.780587 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:39.780575    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fbd6f7b-2139-4134-9577-fce124537c73-samples-operator-tls podName:9fbd6f7b-2139-4134-9577-fce124537c73 nodeName:}" failed. No retries permitted until 2026-09-17 14:05:43.780558852 +0000 UTC m=+44.782102310 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9fbd6f7b-2139-4134-9577-fce124537c73-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-47zgn" (UID: "9fbd6f7b-2139-4134-9577-fce124537c73") : secret "samples-operator-tls" not found
Sep 17 14:05:39.881763 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:39.881685    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b9b37579-a215-4a77-bd0a-f33675b1b17f-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-7hhfj\" (UID: \"b9b37579-a215-4a77-bd0a-f33675b1b17f\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-7hhfj"
Sep 17 14:05:39.881889 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:39.881843    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 17 14:05:39.881932 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:39.881909    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9b37579-a215-4a77-bd0a-f33675b1b17f-networking-console-plugin-cert podName:b9b37579-a215-4a77-bd0a-f33675b1b17f nodeName:}" failed. No retries permitted until 2026-09-17 14:05:43.881893329 +0000 UTC m=+44.883436800 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b9b37579-a215-4a77-bd0a-f33675b1b17f-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-7hhfj" (UID: "b9b37579-a215-4a77-bd0a-f33675b1b17f") : secret "networking-console-plugin-cert" not found
Sep 17 14:05:40.189577 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:40.189489    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:40.295105 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:40.295068    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:40.387520 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:40.387490    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:40.490216 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:40.490180    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:40.691624 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:40.691574    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/81211573-af3a-4b68-ac05-92fdbb7cb8b2-registry-tls\") pod \"image-registry-c7fc9df5d-jl62r\" (UID: \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\") " pod="openshift-image-registry/image-registry-c7fc9df5d-jl62r"
Sep 17 14:05:40.692079 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:40.691640    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/40b77f59-aa77-4958-bd61-2522680f1ecf-metrics-tls\") pod \"dns-default-6v652\" (UID: \"40b77f59-aa77-4958-bd61-2522680f1ecf\") " pod="openshift-dns/dns-default-6v652"
Sep 17 14:05:40.692079 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:40.691766    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 14:05:40.692079 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:40.691772    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 14:05:40.692079 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:40.691795    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c7fc9df5d-jl62r: secret "image-registry-tls" not found
Sep 17 14:05:40.692079 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:40.691853    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/81211573-af3a-4b68-ac05-92fdbb7cb8b2-registry-tls podName:81211573-af3a-4b68-ac05-92fdbb7cb8b2 nodeName:}" failed. No retries permitted until 2026-09-17 14:05:48.691833165 +0000 UTC m=+49.693376634 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/81211573-af3a-4b68-ac05-92fdbb7cb8b2-registry-tls") pod "image-registry-c7fc9df5d-jl62r" (UID: "81211573-af3a-4b68-ac05-92fdbb7cb8b2") : secret "image-registry-tls" not found
Sep 17 14:05:40.692079 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:40.691903    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40b77f59-aa77-4958-bd61-2522680f1ecf-metrics-tls podName:40b77f59-aa77-4958-bd61-2522680f1ecf nodeName:}" failed. No retries permitted until 2026-09-17 14:05:48.69187419 +0000 UTC m=+49.693417647 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/40b77f59-aa77-4958-bd61-2522680f1ecf-metrics-tls") pod "dns-default-6v652" (UID: "40b77f59-aa77-4958-bd61-2522680f1ecf") : secret "dns-default-metrics-tls" not found
Sep 17 14:05:42.477324 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:42.477292    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:42.655567 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:42.655536    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:43.245290 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:43.245208    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:43.398735 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:43.398704    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:43.620942 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:43.620837    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1414b9c2-7c96-4383-9947-e7abcd68a074-metrics-certs\") pod \"router-default-68854cb984-hm4cb\" (UID: \"1414b9c2-7c96-4383-9947-e7abcd68a074\") " pod="openshift-ingress/router-default-68854cb984-hm4cb"
Sep 17 14:05:43.621473 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:43.620946    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1414b9c2-7c96-4383-9947-e7abcd68a074-service-ca-bundle\") pod \"router-default-68854cb984-hm4cb\" (UID: \"1414b9c2-7c96-4383-9947-e7abcd68a074\") " pod="openshift-ingress/router-default-68854cb984-hm4cb"
Sep 17 14:05:43.621473 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:43.621021    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 17 14:05:43.621473 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:43.621101    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1414b9c2-7c96-4383-9947-e7abcd68a074-metrics-certs podName:1414b9c2-7c96-4383-9947-e7abcd68a074 nodeName:}" failed. No retries permitted until 2026-09-17 14:05:51.62108113 +0000 UTC m=+52.622624597 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1414b9c2-7c96-4383-9947-e7abcd68a074-metrics-certs") pod "router-default-68854cb984-hm4cb" (UID: "1414b9c2-7c96-4383-9947-e7abcd68a074") : secret "router-metrics-certs-default" not found
Sep 17 14:05:43.621473 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:43.621133    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1414b9c2-7c96-4383-9947-e7abcd68a074-service-ca-bundle podName:1414b9c2-7c96-4383-9947-e7abcd68a074 nodeName:}" failed. No retries permitted until 2026-09-17 14:05:51.621114525 +0000 UTC m=+52.622657981 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1414b9c2-7c96-4383-9947-e7abcd68a074-service-ca-bundle") pod "router-default-68854cb984-hm4cb" (UID: "1414b9c2-7c96-4383-9947-e7abcd68a074") : configmap references non-existent config key: service-ca.crt
Sep 17 14:05:43.721693 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:43.721649    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-r58pd\" (UID: \"1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-r58pd"
Sep 17 14:05:43.721887 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:43.721860    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 17 14:05:43.722125 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:43.722112    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99-cluster-monitoring-operator-tls podName:1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99 nodeName:}" failed. No retries permitted until 2026-09-17 14:05:51.721917585 +0000 UTC m=+52.723461055 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-r58pd" (UID: "1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99") : secret "cluster-monitoring-operator-tls" not found
Sep 17 14:05:43.724578 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:43.724555    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-rr97k_45ffa13d-635a-46e8-8894-30902b9cf719/console-operator/0.log"
Sep 17 14:05:43.724766 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:43.724594    2582 generic.go:356] "Generic (PLEG): container finished" podID="45ffa13d-635a-46e8-8894-30902b9cf719" containerID="d9d04762c6a0eece2bbd3fe95a63152a23a91105448f38fb66095958ff0497a9" exitCode=255
Sep 17 14:05:43.724766 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:43.724656    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-rr97k" event={"ID":"45ffa13d-635a-46e8-8894-30902b9cf719","Type":"ContainerDied","Data":"d9d04762c6a0eece2bbd3fe95a63152a23a91105448f38fb66095958ff0497a9"}
Sep 17 14:05:43.724956 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:43.724942    2582 scope.go:117] "RemoveContainer" containerID="d9d04762c6a0eece2bbd3fe95a63152a23a91105448f38fb66095958ff0497a9"
Sep 17 14:05:43.727196 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:43.727169    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wxbtg" event={"ID":"9857487e-aa03-45ac-88e4-e6a40b699fef","Type":"ContainerStarted","Data":"f857606855ed2c3128ca7e15faf31cd915185c35137e37d36b1d13d6c4dd18cc"}
Sep 17 14:05:43.729358 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:43.729330    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-thrb2" event={"ID":"903d323b-ec37-4e01-89e9-7ab8caa061d7","Type":"ContainerStarted","Data":"c61bae455811e4033751a0fe9d4a391241a0a097946f3155f89a93f02ef262ee"}
Sep 17 14:05:43.731104 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:43.731045    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-dhfb8" event={"ID":"83ecb87d-85fa-4ac1-9800-103030f2d959","Type":"ContainerStarted","Data":"3fe562d1a564e4c77ca39b0df5c016f493f202282b6ac7d5021fa59e8247fc28"}
Sep 17 14:05:43.732773 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:43.732729    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jl6xc" event={"ID":"a51d828f-334b-4be5-93f3-33eabaa37533","Type":"ContainerStarted","Data":"9db16563a02168b3f4564c9f6ae295720da4a4e94effaba3ff2a044d1cbae7cf"}
Sep 17 14:05:43.736189 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:43.736165    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-94gvr" event={"ID":"c93fff25-711a-4af0-90d8-08368c6161da","Type":"ContainerStarted","Data":"86558e60476122359b2b35122944fa1597bc36d3b96937ef1e27fe72b58a384c"}
Sep 17 14:05:43.783945 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:43.783889    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5844b856db-thrb2" podStartSLOduration=2.609071749 podStartE2EDuration="8.783868153s" totalImagesPullingTime="6.174796404s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:05:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:05:43.782666615 +0000 UTC m=+44.784210088" watchObservedRunningTime="2026-09-17 14:05:43.783868153 +0000 UTC m=+44.785411627"
Sep 17 14:05:43.806899 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:43.806779    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-dhfb8" podStartSLOduration=2.78242745 podStartE2EDuration="8.806744027s" totalImagesPullingTime="6.024316577s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:05:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:05:43.806223272 +0000 UTC m=+44.807766748" watchObservedRunningTime="2026-09-17 14:05:43.806744027 +0000 UTC m=+44.808287503"
Sep 17 14:05:43.824388 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:43.824351    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9fbd6f7b-2139-4134-9577-fce124537c73-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-47zgn\" (UID: \"9fbd6f7b-2139-4134-9577-fce124537c73\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-47zgn"
Sep 17 14:05:43.824753 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:43.824726    2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 17 14:05:43.824890 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:43.824810    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fbd6f7b-2139-4134-9577-fce124537c73-samples-operator-tls podName:9fbd6f7b-2139-4134-9577-fce124537c73 nodeName:}" failed. No retries permitted until 2026-09-17 14:05:51.824788949 +0000 UTC m=+52.826332613 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9fbd6f7b-2139-4134-9577-fce124537c73-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-47zgn" (UID: "9fbd6f7b-2139-4134-9577-fce124537c73") : secret "samples-operator-tls" not found
Sep 17 14:05:43.825343 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:43.825298    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jl6xc" podStartSLOduration=2.899380343 podStartE2EDuration="8.825278081s" totalImagesPullingTime="5.925897738s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:05:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:05:43.824430194 +0000 UTC m=+44.825973669" watchObservedRunningTime="2026-09-17 14:05:43.825278081 +0000 UTC m=+44.826821557"
Sep 17 14:05:43.860683 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:43.860291    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f8b768455-94gvr" podStartSLOduration=1.6326415029999999 podStartE2EDuration="7.86026935s" totalImagesPullingTime="6.227627847s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:05:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:05:43.844749529 +0000 UTC m=+44.846293003" watchObservedRunningTime="2026-09-17 14:05:43.86026935 +0000 UTC m=+44.861812826"
Sep 17 14:05:43.925748 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:43.925714    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b9b37579-a215-4a77-bd0a-f33675b1b17f-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-7hhfj\" (UID: \"b9b37579-a215-4a77-bd0a-f33675b1b17f\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-7hhfj"
Sep 17 14:05:43.925979 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:43.925961    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 17 14:05:43.926062 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:43.926044    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9b37579-a215-4a77-bd0a-f33675b1b17f-networking-console-plugin-cert podName:b9b37579-a215-4a77-bd0a-f33675b1b17f nodeName:}" failed. No retries permitted until 2026-09-17 14:05:51.926015484 +0000 UTC m=+52.927558947 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b9b37579-a215-4a77-bd0a-f33675b1b17f-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-7hhfj" (UID: "b9b37579-a215-4a77-bd0a-f33675b1b17f") : secret "networking-console-plugin-cert" not found
Sep 17 14:05:43.957190 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:43.957146    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:44.149130 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:44.149096    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:44.377952 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:44.377904    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wxbtg" podStartSLOduration=3.457759637 podStartE2EDuration="9.377885084s" totalImagesPullingTime="5.920125447s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:05:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:05:43.861738747 +0000 UTC m=+44.863282223" watchObservedRunningTime="2026-09-17 14:05:44.377885084 +0000 UTC m=+45.379428558"
Sep 17 14:05:44.378502 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:44.378483    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-4dt5r"]
Sep 17 14:05:44.397559 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:44.397531    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4dt5r"
Sep 17 14:05:44.400329 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:44.400300    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-48lqt\""
Sep 17 14:05:44.400542 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:44.400524    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 17 14:05:44.400823 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:44.400802    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 17 14:05:44.400903 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:44.400881    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4dt5r"]
Sep 17 14:05:44.430887 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:44.430848    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5cd9349e-5c41-4588-98e3-35fbe230801c-data-volume\") pod \"insights-runtime-extractor-4dt5r\" (UID: \"5cd9349e-5c41-4588-98e3-35fbe230801c\") " pod="openshift-insights/insights-runtime-extractor-4dt5r"
Sep 17 14:05:44.431061 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:44.430971    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5cd9349e-5c41-4588-98e3-35fbe230801c-crio-socket\") pod \"insights-runtime-extractor-4dt5r\" (UID: \"5cd9349e-5c41-4588-98e3-35fbe230801c\") " pod="openshift-insights/insights-runtime-extractor-4dt5r"
Sep 17 14:05:44.431061 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:44.431014    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5cd9349e-5c41-4588-98e3-35fbe230801c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4dt5r\" (UID: \"5cd9349e-5c41-4588-98e3-35fbe230801c\") " pod="openshift-insights/insights-runtime-extractor-4dt5r"
Sep 17 14:05:44.431205 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:44.431068    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmck4\" (UniqueName: \"kubernetes.io/projected/5cd9349e-5c41-4588-98e3-35fbe230801c-kube-api-access-lmck4\") pod \"insights-runtime-extractor-4dt5r\" (UID: \"5cd9349e-5c41-4588-98e3-35fbe230801c\") " pod="openshift-insights/insights-runtime-extractor-4dt5r"
Sep 17 14:05:44.431205 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:44.431094    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5cd9349e-5c41-4588-98e3-35fbe230801c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4dt5r\" (UID: \"5cd9349e-5c41-4588-98e3-35fbe230801c\") " pod="openshift-insights/insights-runtime-extractor-4dt5r"
Sep 17 14:05:44.532467 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:44.532426    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5cd9349e-5c41-4588-98e3-35fbe230801c-data-volume\") pod \"insights-runtime-extractor-4dt5r\" (UID: \"5cd9349e-5c41-4588-98e3-35fbe230801c\") " pod="openshift-insights/insights-runtime-extractor-4dt5r"
Sep 17 14:05:44.532680 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:44.532483    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5cd9349e-5c41-4588-98e3-35fbe230801c-crio-socket\") pod \"insights-runtime-extractor-4dt5r\" (UID: \"5cd9349e-5c41-4588-98e3-35fbe230801c\") " pod="openshift-insights/insights-runtime-extractor-4dt5r"
Sep 17 14:05:44.532680 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:44.532501    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5cd9349e-5c41-4588-98e3-35fbe230801c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4dt5r\" (UID: \"5cd9349e-5c41-4588-98e3-35fbe230801c\") " pod="openshift-insights/insights-runtime-extractor-4dt5r"
Sep 17 14:05:44.532680 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:44.532533    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmck4\" (UniqueName: \"kubernetes.io/projected/5cd9349e-5c41-4588-98e3-35fbe230801c-kube-api-access-lmck4\") pod \"insights-runtime-extractor-4dt5r\" (UID: \"5cd9349e-5c41-4588-98e3-35fbe230801c\") " pod="openshift-insights/insights-runtime-extractor-4dt5r"
Sep 17 14:05:44.532680 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:44.532552    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5cd9349e-5c41-4588-98e3-35fbe230801c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4dt5r\" (UID: \"5cd9349e-5c41-4588-98e3-35fbe230801c\") " pod="openshift-insights/insights-runtime-extractor-4dt5r"
Sep 17 14:05:44.532680 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:44.532668    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 14:05:44.532942 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:44.532700    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 14:05:44.532942 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:44.532702    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5cd9349e-5c41-4588-98e3-35fbe230801c-crio-socket\") pod \"insights-runtime-extractor-4dt5r\" (UID: \"5cd9349e-5c41-4588-98e3-35fbe230801c\") " pod="openshift-insights/insights-runtime-extractor-4dt5r"
Sep 17 14:05:44.532942 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:44.532747    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5cd9349e-5c41-4588-98e3-35fbe230801c-kube-rbac-proxy-cm podName:5cd9349e-5c41-4588-98e3-35fbe230801c nodeName:}" failed. No retries permitted until 2026-09-17 14:05:45.032725343 +0000 UTC m=+46.034268812 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5cd9349e-5c41-4588-98e3-35fbe230801c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4dt5r" (UID: "5cd9349e-5c41-4588-98e3-35fbe230801c") : configmap "kube-rbac-proxy" not found
Sep 17 14:05:44.532942 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:44.532761    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cd9349e-5c41-4588-98e3-35fbe230801c-insights-runtime-extractor-tls podName:5cd9349e-5c41-4588-98e3-35fbe230801c nodeName:}" failed. No retries permitted until 2026-09-17 14:05:45.03275549 +0000 UTC m=+46.034298941 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5cd9349e-5c41-4588-98e3-35fbe230801c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4dt5r" (UID: "5cd9349e-5c41-4588-98e3-35fbe230801c") : secret "insights-runtime-extractor-tls" not found
Sep 17 14:05:44.532942 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:44.532806    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5cd9349e-5c41-4588-98e3-35fbe230801c-data-volume\") pod \"insights-runtime-extractor-4dt5r\" (UID: \"5cd9349e-5c41-4588-98e3-35fbe230801c\") " pod="openshift-insights/insights-runtime-extractor-4dt5r"
Sep 17 14:05:44.541811 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:44.541779    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmck4\" (UniqueName: \"kubernetes.io/projected/5cd9349e-5c41-4588-98e3-35fbe230801c-kube-api-access-lmck4\") pod \"insights-runtime-extractor-4dt5r\" (UID: \"5cd9349e-5c41-4588-98e3-35fbe230801c\") " pod="openshift-insights/insights-runtime-extractor-4dt5r"
Sep 17 14:05:44.599331 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:44.599293    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:44.741224 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:44.741193    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-rr97k_45ffa13d-635a-46e8-8894-30902b9cf719/console-operator/1.log"
Sep 17 14:05:44.741733 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:44.741379    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:44.741733 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:44.741584    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-rr97k_45ffa13d-635a-46e8-8894-30902b9cf719/console-operator/0.log"
Sep 17 14:05:44.741733 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:44.741634    2582 generic.go:356] "Generic (PLEG): container finished" podID="45ffa13d-635a-46e8-8894-30902b9cf719" containerID="2bc3ede0b7451dedd4633a3393fd1c5bd8deb7d58c84700564bb9b6a272332f4" exitCode=255
Sep 17 14:05:44.741733 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:44.741662    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-rr97k" event={"ID":"45ffa13d-635a-46e8-8894-30902b9cf719","Type":"ContainerDied","Data":"2bc3ede0b7451dedd4633a3393fd1c5bd8deb7d58c84700564bb9b6a272332f4"}
Sep 17 14:05:44.741733 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:44.741723    2582 scope.go:117] "RemoveContainer" containerID="d9d04762c6a0eece2bbd3fe95a63152a23a91105448f38fb66095958ff0497a9"
Sep 17 14:05:44.741984 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:44.741968    2582 scope.go:117] "RemoveContainer" containerID="2bc3ede0b7451dedd4633a3393fd1c5bd8deb7d58c84700564bb9b6a272332f4"
Sep 17 14:05:44.742176 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:44.742155    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-rr97k_openshift-console-operator(45ffa13d-635a-46e8-8894-30902b9cf719)\"" pod="openshift-console-operator/console-operator-696884fb9c-rr97k" podUID="45ffa13d-635a-46e8-8894-30902b9cf719"
Sep 17 14:05:45.038298 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:45.038209    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5cd9349e-5c41-4588-98e3-35fbe230801c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4dt5r\" (UID: \"5cd9349e-5c41-4588-98e3-35fbe230801c\") " pod="openshift-insights/insights-runtime-extractor-4dt5r"
Sep 17 14:05:45.038298 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:45.038268    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5cd9349e-5c41-4588-98e3-35fbe230801c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4dt5r\" (UID: \"5cd9349e-5c41-4588-98e3-35fbe230801c\") " pod="openshift-insights/insights-runtime-extractor-4dt5r"
Sep 17 14:05:45.038474 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:45.038334    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 14:05:45.038474 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:45.038368    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 14:05:45.038474 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:45.038404    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5cd9349e-5c41-4588-98e3-35fbe230801c-kube-rbac-proxy-cm podName:5cd9349e-5c41-4588-98e3-35fbe230801c nodeName:}" failed. No retries permitted until 2026-09-17 14:05:46.038388769 +0000 UTC m=+47.039932224 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5cd9349e-5c41-4588-98e3-35fbe230801c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4dt5r" (UID: "5cd9349e-5c41-4588-98e3-35fbe230801c") : configmap "kube-rbac-proxy" not found
Sep 17 14:05:45.038474 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:45.038418    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cd9349e-5c41-4588-98e3-35fbe230801c-insights-runtime-extractor-tls podName:5cd9349e-5c41-4588-98e3-35fbe230801c nodeName:}" failed. No retries permitted until 2026-09-17 14:05:46.038412367 +0000 UTC m=+47.039955819 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5cd9349e-5c41-4588-98e3-35fbe230801c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4dt5r" (UID: "5cd9349e-5c41-4588-98e3-35fbe230801c") : secret "insights-runtime-extractor-tls" not found
Sep 17 14:05:45.139014 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:45.138983    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:45.292667 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:45.292580    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:45.435314 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:45.435277    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:45.746328 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:45.746302    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-rr97k_45ffa13d-635a-46e8-8894-30902b9cf719/console-operator/1.log"
Sep 17 14:05:45.746740 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:45.746648    2582 scope.go:117] "RemoveContainer" containerID="2bc3ede0b7451dedd4633a3393fd1c5bd8deb7d58c84700564bb9b6a272332f4"
Sep 17 14:05:45.746805 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:45.746787    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-rr97k_openshift-console-operator(45ffa13d-635a-46e8-8894-30902b9cf719)\"" pod="openshift-console-operator/console-operator-696884fb9c-rr97k" podUID="45ffa13d-635a-46e8-8894-30902b9cf719"
Sep 17 14:05:46.046742 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:46.046645    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5cd9349e-5c41-4588-98e3-35fbe230801c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4dt5r\" (UID: \"5cd9349e-5c41-4588-98e3-35fbe230801c\") " pod="openshift-insights/insights-runtime-extractor-4dt5r"
Sep 17 14:05:46.046920 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:46.046789    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5cd9349e-5c41-4588-98e3-35fbe230801c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4dt5r\" (UID: \"5cd9349e-5c41-4588-98e3-35fbe230801c\") " pod="openshift-insights/insights-runtime-extractor-4dt5r"
Sep 17 14:05:46.046920 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:46.046806    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 14:05:46.046920 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:46.046864    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 14:05:46.046920 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:46.046870    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cd9349e-5c41-4588-98e3-35fbe230801c-insights-runtime-extractor-tls podName:5cd9349e-5c41-4588-98e3-35fbe230801c nodeName:}" failed. No retries permitted until 2026-09-17 14:05:48.046854461 +0000 UTC m=+49.048397913 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5cd9349e-5c41-4588-98e3-35fbe230801c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4dt5r" (UID: "5cd9349e-5c41-4588-98e3-35fbe230801c") : secret "insights-runtime-extractor-tls" not found
Sep 17 14:05:46.046920 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:46.046902    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5cd9349e-5c41-4588-98e3-35fbe230801c-kube-rbac-proxy-cm podName:5cd9349e-5c41-4588-98e3-35fbe230801c nodeName:}" failed. No retries permitted until 2026-09-17 14:05:48.046891021 +0000 UTC m=+49.048434491 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5cd9349e-5c41-4588-98e3-35fbe230801c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4dt5r" (UID: "5cd9349e-5c41-4588-98e3-35fbe230801c") : configmap "kube-rbac-proxy" not found
Sep 17 14:05:46.206847 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:46.206798    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-rr97k"
Sep 17 14:05:46.207014 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:46.206867    2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-rr97k"
Sep 17 14:05:46.492892 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:46.492864    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-s2nlk_1268d6c2-0f80-41e0-98d5-d72d5c03da1b/dns-node-resolver/0.log"
Sep 17 14:05:46.750152 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:46.750078    2582 scope.go:117] "RemoveContainer" containerID="2bc3ede0b7451dedd4633a3393fd1c5bd8deb7d58c84700564bb9b6a272332f4"
Sep 17 14:05:46.750526 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:46.750248    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-rr97k_openshift-console-operator(45ffa13d-635a-46e8-8894-30902b9cf719)\"" pod="openshift-console-operator/console-operator-696884fb9c-rr97k" podUID="45ffa13d-635a-46e8-8894-30902b9cf719"
Sep 17 14:05:47.083156 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:47.083082    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:47.093522 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:47.093495    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-72skg_4462f6aa-ccc5-42d0-8623-f0dad6105560/node-ca/0.log"
Sep 17 14:05:47.304790 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:47.304752    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:47.958484 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:47.958440    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:48.065344 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:48.065299    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5cd9349e-5c41-4588-98e3-35fbe230801c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4dt5r\" (UID: \"5cd9349e-5c41-4588-98e3-35fbe230801c\") " pod="openshift-insights/insights-runtime-extractor-4dt5r"
Sep 17 14:05:48.065546 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:48.065357    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5cd9349e-5c41-4588-98e3-35fbe230801c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4dt5r\" (UID: \"5cd9349e-5c41-4588-98e3-35fbe230801c\") " pod="openshift-insights/insights-runtime-extractor-4dt5r"
Sep 17 14:05:48.065546 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:48.065442    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 14:05:48.065546 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:48.065486    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 14:05:48.065546 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:48.065537    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5cd9349e-5c41-4588-98e3-35fbe230801c-kube-rbac-proxy-cm podName:5cd9349e-5c41-4588-98e3-35fbe230801c nodeName:}" failed. No retries permitted until 2026-09-17 14:05:52.065516301 +0000 UTC m=+53.067059769 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5cd9349e-5c41-4588-98e3-35fbe230801c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4dt5r" (UID: "5cd9349e-5c41-4588-98e3-35fbe230801c") : configmap "kube-rbac-proxy" not found
Sep 17 14:05:48.065812 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:48.065558    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cd9349e-5c41-4588-98e3-35fbe230801c-insights-runtime-extractor-tls podName:5cd9349e-5c41-4588-98e3-35fbe230801c nodeName:}" failed. No retries permitted until 2026-09-17 14:05:52.065548865 +0000 UTC m=+53.067092317 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5cd9349e-5c41-4588-98e3-35fbe230801c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4dt5r" (UID: "5cd9349e-5c41-4588-98e3-35fbe230801c") : secret "insights-runtime-extractor-tls" not found
Sep 17 14:05:48.295530 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:48.295450    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zjz6h_5cd5a40b-4883-4928-8dd6-b0c3d13ad8ed/serve-healthcheck-canary/0.log"
Sep 17 14:05:48.772540 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:48.772504    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/81211573-af3a-4b68-ac05-92fdbb7cb8b2-registry-tls\") pod \"image-registry-c7fc9df5d-jl62r\" (UID: \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\") " pod="openshift-image-registry/image-registry-c7fc9df5d-jl62r"
Sep 17 14:05:48.772540 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:48.772543    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/40b77f59-aa77-4958-bd61-2522680f1ecf-metrics-tls\") pod \"dns-default-6v652\" (UID: \"40b77f59-aa77-4958-bd61-2522680f1ecf\") " pod="openshift-dns/dns-default-6v652"
Sep 17 14:05:48.772836 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:48.772694    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 14:05:48.772836 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:48.772714    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c7fc9df5d-jl62r: secret "image-registry-tls" not found
Sep 17 14:05:48.772836 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:48.772736    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 14:05:48.772836 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:48.772781    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/81211573-af3a-4b68-ac05-92fdbb7cb8b2-registry-tls podName:81211573-af3a-4b68-ac05-92fdbb7cb8b2 nodeName:}" failed. No retries permitted until 2026-09-17 14:06:04.772760395 +0000 UTC m=+65.774303874 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/81211573-af3a-4b68-ac05-92fdbb7cb8b2-registry-tls") pod "image-registry-c7fc9df5d-jl62r" (UID: "81211573-af3a-4b68-ac05-92fdbb7cb8b2") : secret "image-registry-tls" not found
Sep 17 14:05:48.772836 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:48.772800    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40b77f59-aa77-4958-bd61-2522680f1ecf-metrics-tls podName:40b77f59-aa77-4958-bd61-2522680f1ecf nodeName:}" failed. No retries permitted until 2026-09-17 14:06:04.772791045 +0000 UTC m=+65.774334499 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/40b77f59-aa77-4958-bd61-2522680f1ecf-metrics-tls") pod "dns-default-6v652" (UID: "40b77f59-aa77-4958-bd61-2522680f1ecf") : secret "dns-default-metrics-tls" not found
Sep 17 14:05:50.658007 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:50.657975    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:51.099666 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:51.099632    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:51.698199 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:51.698155    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1414b9c2-7c96-4383-9947-e7abcd68a074-service-ca-bundle\") pod \"router-default-68854cb984-hm4cb\" (UID: \"1414b9c2-7c96-4383-9947-e7abcd68a074\") " pod="openshift-ingress/router-default-68854cb984-hm4cb"
Sep 17 14:05:51.698702 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:51.698349    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1414b9c2-7c96-4383-9947-e7abcd68a074-service-ca-bundle podName:1414b9c2-7c96-4383-9947-e7abcd68a074 nodeName:}" failed. No retries permitted until 2026-09-17 14:06:07.698329081 +0000 UTC m=+68.699872547 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1414b9c2-7c96-4383-9947-e7abcd68a074-service-ca-bundle") pod "router-default-68854cb984-hm4cb" (UID: "1414b9c2-7c96-4383-9947-e7abcd68a074") : configmap references non-existent config key: service-ca.crt
Sep 17 14:05:51.698702 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:51.698442    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1414b9c2-7c96-4383-9947-e7abcd68a074-metrics-certs\") pod \"router-default-68854cb984-hm4cb\" (UID: \"1414b9c2-7c96-4383-9947-e7abcd68a074\") " pod="openshift-ingress/router-default-68854cb984-hm4cb"
Sep 17 14:05:51.698845 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:51.698753    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 17 14:05:51.698845 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:51.698841    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1414b9c2-7c96-4383-9947-e7abcd68a074-metrics-certs podName:1414b9c2-7c96-4383-9947-e7abcd68a074 nodeName:}" failed. No retries permitted until 2026-09-17 14:06:07.698822992 +0000 UTC m=+68.700366448 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1414b9c2-7c96-4383-9947-e7abcd68a074-metrics-certs") pod "router-default-68854cb984-hm4cb" (UID: "1414b9c2-7c96-4383-9947-e7abcd68a074") : secret "router-metrics-certs-default" not found
Sep 17 14:05:51.799001 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:51.798959    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-r58pd\" (UID: \"1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-r58pd"
Sep 17 14:05:51.799191 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:51.799123    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 17 14:05:51.799271 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:51.799214    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99-cluster-monitoring-operator-tls podName:1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99 nodeName:}" failed. No retries permitted until 2026-09-17 14:06:07.799195644 +0000 UTC m=+68.800739115 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-r58pd" (UID: "1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99") : secret "cluster-monitoring-operator-tls" not found
Sep 17 14:05:51.856868 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:51.856827    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:51.900329 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:51.900295    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9fbd6f7b-2139-4134-9577-fce124537c73-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-47zgn\" (UID: \"9fbd6f7b-2139-4134-9577-fce124537c73\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-47zgn"
Sep 17 14:05:51.900498 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:51.900473    2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 17 14:05:51.900564 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:51.900550    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fbd6f7b-2139-4134-9577-fce124537c73-samples-operator-tls podName:9fbd6f7b-2139-4134-9577-fce124537c73 nodeName:}" failed. No retries permitted until 2026-09-17 14:06:07.900529129 +0000 UTC m=+68.902072600 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9fbd6f7b-2139-4134-9577-fce124537c73-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-47zgn" (UID: "9fbd6f7b-2139-4134-9577-fce124537c73") : secret "samples-operator-tls" not found
Sep 17 14:05:52.004920 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:52.004834    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b9b37579-a215-4a77-bd0a-f33675b1b17f-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-7hhfj\" (UID: \"b9b37579-a215-4a77-bd0a-f33675b1b17f\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-7hhfj"
Sep 17 14:05:52.005061 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:52.004992    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 17 14:05:52.005100 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:52.005066    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9b37579-a215-4a77-bd0a-f33675b1b17f-networking-console-plugin-cert podName:b9b37579-a215-4a77-bd0a-f33675b1b17f nodeName:}" failed. No retries permitted until 2026-09-17 14:06:08.005050297 +0000 UTC m=+69.006593749 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b9b37579-a215-4a77-bd0a-f33675b1b17f-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-7hhfj" (UID: "b9b37579-a215-4a77-bd0a-f33675b1b17f") : secret "networking-console-plugin-cert" not found
Sep 17 14:05:52.105973 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:52.105932    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5cd9349e-5c41-4588-98e3-35fbe230801c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4dt5r\" (UID: \"5cd9349e-5c41-4588-98e3-35fbe230801c\") " pod="openshift-insights/insights-runtime-extractor-4dt5r"
Sep 17 14:05:52.106157 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:52.105998    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5cd9349e-5c41-4588-98e3-35fbe230801c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4dt5r\" (UID: \"5cd9349e-5c41-4588-98e3-35fbe230801c\") " pod="openshift-insights/insights-runtime-extractor-4dt5r"
Sep 17 14:05:52.106157 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:52.106108    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 14:05:52.106250 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:05:52.106167    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cd9349e-5c41-4588-98e3-35fbe230801c-insights-runtime-extractor-tls podName:5cd9349e-5c41-4588-98e3-35fbe230801c nodeName:}" failed. No retries permitted until 2026-09-17 14:06:00.106153546 +0000 UTC m=+61.107696997 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5cd9349e-5c41-4588-98e3-35fbe230801c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4dt5r" (UID: "5cd9349e-5c41-4588-98e3-35fbe230801c") : secret "insights-runtime-extractor-tls" not found
Sep 17 14:05:52.106511 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:52.106491    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5cd9349e-5c41-4588-98e3-35fbe230801c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4dt5r\" (UID: \"5cd9349e-5c41-4588-98e3-35fbe230801c\") " pod="openshift-insights/insights-runtime-extractor-4dt5r"
Sep 17 14:05:52.187654 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:52.187597    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:52.353221 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:52.353115    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:52.567237 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:52.567187    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:53.441635 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:53.441594    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:53.613505 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:53.613468    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:53.915458 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:53.915431    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:54.116678 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:54.116649    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:54.261473 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:54.261442    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:54.549189 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:54.549111    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:55.314893 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:55.314861    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:55.513044 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:55.513015    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:55.674171 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:55.674095    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:55.826217 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:55.826188    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:55.968543 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:55.968495    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:56.152118 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:56.152085    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:56.313524 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:56.313444    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:56.442538 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:56.442506    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:57.333452 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:57.333419    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:57.533307 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:57.533261    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:57.902417 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:57.902382    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:58.020907 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:58.020866    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:58.142116 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:58.142085    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:58.514533 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:58.514503    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:58.655155 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:58.655123    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qn4cf"
Sep 17 14:05:59.406132 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:59.406084    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:05:59.842992 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:05:59.842960    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:00.047888 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:00.047857    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:00.181210 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:00.181116    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5cd9349e-5c41-4588-98e3-35fbe230801c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4dt5r\" (UID: \"5cd9349e-5c41-4588-98e3-35fbe230801c\") " pod="openshift-insights/insights-runtime-extractor-4dt5r"
Sep 17 14:06:00.183417 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:00.183396    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5cd9349e-5c41-4588-98e3-35fbe230801c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4dt5r\" (UID: \"5cd9349e-5c41-4588-98e3-35fbe230801c\") " pod="openshift-insights/insights-runtime-extractor-4dt5r"
Sep 17 14:06:00.252930 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:00.252899    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:00.312946 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:00.312913    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-48lqt\""
Sep 17 14:06:00.321249 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:00.321217    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4dt5r"
Sep 17 14:06:00.487114 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:00.487084    2582 scope.go:117] "RemoveContainer" containerID="2bc3ede0b7451dedd4633a3393fd1c5bd8deb7d58c84700564bb9b6a272332f4"
Sep 17 14:06:00.599494 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:00.599471    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4dt5r"]
Sep 17 14:06:00.619812 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:00.619781    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:00.632501 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:06:00.632461    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cd9349e_5c41_4588_98e3_35fbe230801c.slice/crio-fa922d6b0c6b6ff5968eb406ba2faf09ebd29dd116a44d6321fe7ed451d715c6 WatchSource:0}: Error finding container fa922d6b0c6b6ff5968eb406ba2faf09ebd29dd116a44d6321fe7ed451d715c6: Status 404 returned error can't find the container with id fa922d6b0c6b6ff5968eb406ba2faf09ebd29dd116a44d6321fe7ed451d715c6
Sep 17 14:06:00.796680 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:00.796568    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4dt5r" event={"ID":"5cd9349e-5c41-4588-98e3-35fbe230801c","Type":"ContainerStarted","Data":"ba44e1d1acc8b433ae6341d447a3c14decb7d7009cdb6288e268d4493d6cda25"}
Sep 17 14:06:00.796680 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:00.796658    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4dt5r" event={"ID":"5cd9349e-5c41-4588-98e3-35fbe230801c","Type":"ContainerStarted","Data":"fa922d6b0c6b6ff5968eb406ba2faf09ebd29dd116a44d6321fe7ed451d715c6"}
Sep 17 14:06:00.798091 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:00.798074    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-rr97k_45ffa13d-635a-46e8-8894-30902b9cf719/console-operator/2.log"
Sep 17 14:06:00.798515 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:00.798500    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-rr97k_45ffa13d-635a-46e8-8894-30902b9cf719/console-operator/1.log"
Sep 17 14:06:00.798579 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:00.798531    2582 generic.go:356] "Generic (PLEG): container finished" podID="45ffa13d-635a-46e8-8894-30902b9cf719" containerID="38cac992b3d877cea1215d81be03306d6b9879d7b116edc03daae89cbdd280de" exitCode=255
Sep 17 14:06:00.798628 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:00.798587    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-rr97k" event={"ID":"45ffa13d-635a-46e8-8894-30902b9cf719","Type":"ContainerDied","Data":"38cac992b3d877cea1215d81be03306d6b9879d7b116edc03daae89cbdd280de"}
Sep 17 14:06:00.798671 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:00.798644    2582 scope.go:117] "RemoveContainer" containerID="2bc3ede0b7451dedd4633a3393fd1c5bd8deb7d58c84700564bb9b6a272332f4"
Sep 17 14:06:00.798956 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:00.798941    2582 scope.go:117] "RemoveContainer" containerID="38cac992b3d877cea1215d81be03306d6b9879d7b116edc03daae89cbdd280de"
Sep 17 14:06:00.799123 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:06:00.799105    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-696884fb9c-rr97k_openshift-console-operator(45ffa13d-635a-46e8-8894-30902b9cf719)\"" pod="openshift-console-operator/console-operator-696884fb9c-rr97k" podUID="45ffa13d-635a-46e8-8894-30902b9cf719"
Sep 17 14:06:01.342349 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:01.342321    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:01.527827 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:01.527791    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:01.803804 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:01.803768    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4dt5r" event={"ID":"5cd9349e-5c41-4588-98e3-35fbe230801c","Type":"ContainerStarted","Data":"7e108605509d79e1ea4ed915dcd473193afeb735f8e536cbbecd2dc51b1484e4"}
Sep 17 14:06:01.805284 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:01.805266    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-rr97k_45ffa13d-635a-46e8-8894-30902b9cf719/console-operator/2.log"
Sep 17 14:06:01.906455 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:01.906422    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:02.111470 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:02.111385    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:02.451237 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:02.451201    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:03.813384 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:03.813350    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4dt5r" event={"ID":"5cd9349e-5c41-4588-98e3-35fbe230801c","Type":"ContainerStarted","Data":"d905b76bd135cf65f90d745f8b1c652c98beec6496cf81a030c9daa7854381ec"}
Sep 17 14:06:03.836980 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:03.836932    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-4dt5r" podStartSLOduration=17.339218785 podStartE2EDuration="19.83691291s" totalImagesPullingTime="2.497694125s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 14:05:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:06:03.835713325 +0000 UTC m=+64.837256799" watchObservedRunningTime="2026-09-17 14:06:03.83691291 +0000 UTC m=+64.838456379"
Sep 17 14:06:04.821764 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:04.821726    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/81211573-af3a-4b68-ac05-92fdbb7cb8b2-registry-tls\") pod \"image-registry-c7fc9df5d-jl62r\" (UID: \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\") " pod="openshift-image-registry/image-registry-c7fc9df5d-jl62r"
Sep 17 14:06:04.821764 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:04.821761    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/40b77f59-aa77-4958-bd61-2522680f1ecf-metrics-tls\") pod \"dns-default-6v652\" (UID: \"40b77f59-aa77-4958-bd61-2522680f1ecf\") " pod="openshift-dns/dns-default-6v652"
Sep 17 14:06:04.824134 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:04.824111    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/40b77f59-aa77-4958-bd61-2522680f1ecf-metrics-tls\") pod \"dns-default-6v652\" (UID: \"40b77f59-aa77-4958-bd61-2522680f1ecf\") " pod="openshift-dns/dns-default-6v652"
Sep 17 14:06:04.824198 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:04.824172    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/81211573-af3a-4b68-ac05-92fdbb7cb8b2-registry-tls\") pod \"image-registry-c7fc9df5d-jl62r\" (UID: \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\") " pod="openshift-image-registry/image-registry-c7fc9df5d-jl62r"
Sep 17 14:06:04.992489 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:04.992454    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-sjbd6\""
Sep 17 14:06:05.000466 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:05.000442    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c7fc9df5d-jl62r"
Sep 17 14:06:05.024228 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:05.024203    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hcwc9\""
Sep 17 14:06:05.032298 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:05.032271    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6v652"
Sep 17 14:06:05.123788 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:05.123759    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31e72879-bbfc-4684-bcbb-fd79b4eec4b6-original-pull-secret\") pod \"global-pull-secret-syncer-7k2wk\" (UID: \"31e72879-bbfc-4684-bcbb-fd79b4eec4b6\") " pod="kube-system/global-pull-secret-syncer-7k2wk"
Sep 17 14:06:05.123946 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:05.123858    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/059afb43-5e5e-4db9-bfa2-d874291c1c3a-metrics-certs\") pod \"network-metrics-daemon-r4p4g\" (UID: \"059afb43-5e5e-4db9-bfa2-d874291c1c3a\") " pod="openshift-multus/network-metrics-daemon-r4p4g"
Sep 17 14:06:05.126353 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:05.126335    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 17 14:06:05.126439 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:05.126412    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 17 14:06:05.136447 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:05.136424    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31e72879-bbfc-4684-bcbb-fd79b4eec4b6-original-pull-secret\") pod \"global-pull-secret-syncer-7k2wk\" (UID: \"31e72879-bbfc-4684-bcbb-fd79b4eec4b6\") " pod="kube-system/global-pull-secret-syncer-7k2wk"
Sep 17 14:06:05.136556 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:05.136472    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/059afb43-5e5e-4db9-bfa2-d874291c1c3a-metrics-certs\") pod \"network-metrics-daemon-r4p4g\" (UID: \"059afb43-5e5e-4db9-bfa2-d874291c1c3a\") " pod="openshift-multus/network-metrics-daemon-r4p4g"
Sep 17 14:06:05.225021 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:05.224982    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvl8x\" (UniqueName: \"kubernetes.io/projected/a2cbe70b-eb96-4779-9143-b95a1204aae2-kube-api-access-nvl8x\") pod \"network-check-target-dg4gz\" (UID: \"a2cbe70b-eb96-4779-9143-b95a1204aae2\") " pod="openshift-network-diagnostics/network-check-target-dg4gz"
Sep 17 14:06:05.227412 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:05.227392    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvl8x\" (UniqueName: \"kubernetes.io/projected/a2cbe70b-eb96-4779-9143-b95a1204aae2-kube-api-access-nvl8x\") pod \"network-check-target-dg4gz\" (UID: \"a2cbe70b-eb96-4779-9143-b95a1204aae2\") " pod="openshift-network-diagnostics/network-check-target-dg4gz"
Sep 17 14:06:05.292745 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:05.292714    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-c7fc9df5d-jl62r"]
Sep 17 14:06:05.296338 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:06:05.296309    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81211573_af3a_4b68_ac05_92fdbb7cb8b2.slice/crio-df8e453457a8d9a1dd5a74ea035ca5277fcd74ea2a70972d4a3a5e372c402366 WatchSource:0}: Error finding container df8e453457a8d9a1dd5a74ea035ca5277fcd74ea2a70972d4a3a5e372c402366: Status 404 returned error can't find the container with id df8e453457a8d9a1dd5a74ea035ca5277fcd74ea2a70972d4a3a5e372c402366
Sep 17 14:06:05.320695 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:05.320667    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6v652"]
Sep 17 14:06:05.323652 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:06:05.323624    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40b77f59_aa77_4958_bd61_2522680f1ecf.slice/crio-4a0bd0ac8e8a89cf6dbbb4b5f14adc8b66393a2b944edb3041ae98518c679107 WatchSource:0}: Error finding container 4a0bd0ac8e8a89cf6dbbb4b5f14adc8b66393a2b944edb3041ae98518c679107: Status 404 returned error can't find the container with id 4a0bd0ac8e8a89cf6dbbb4b5f14adc8b66393a2b944edb3041ae98518c679107
Sep 17 14:06:05.399232 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:05.399189    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7k2wk"
Sep 17 14:06:05.408758 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:05.408731    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-27m2f\""
Sep 17 14:06:05.413867 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:05.413838    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lkjmx\""
Sep 17 14:06:05.416581 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:05.416563    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r4p4g"
Sep 17 14:06:05.421828 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:05.421802    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dg4gz"
Sep 17 14:06:05.678305 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:05.678281    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7k2wk"]
Sep 17 14:06:05.680848 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:06:05.680826    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31e72879_bbfc_4684_bcbb_fd79b4eec4b6.slice/crio-3f06098386e3a8ba7382928778dde57ae6c80041c801470825b2abe0c9cca414 WatchSource:0}: Error finding container 3f06098386e3a8ba7382928778dde57ae6c80041c801470825b2abe0c9cca414: Status 404 returned error can't find the container with id 3f06098386e3a8ba7382928778dde57ae6c80041c801470825b2abe0c9cca414
Sep 17 14:06:05.705424 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:05.705392    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dg4gz"]
Sep 17 14:06:05.708918 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:06:05.708877    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2cbe70b_eb96_4779_9143_b95a1204aae2.slice/crio-d07c393b74c44d7820ccc6193e3a9a44b55b286adeee050ca41401414eab6a61 WatchSource:0}: Error finding container d07c393b74c44d7820ccc6193e3a9a44b55b286adeee050ca41401414eab6a61: Status 404 returned error can't find the container with id d07c393b74c44d7820ccc6193e3a9a44b55b286adeee050ca41401414eab6a61
Sep 17 14:06:05.710576 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:05.710547    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-r4p4g"]
Sep 17 14:06:05.713580 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:06:05.713558    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod059afb43_5e5e_4db9_bfa2_d874291c1c3a.slice/crio-0681db6835f226d32127ace2627dc5eee121af810dce9c7f46fef8da54509cee WatchSource:0}: Error finding container 0681db6835f226d32127ace2627dc5eee121af810dce9c7f46fef8da54509cee: Status 404 returned error can't find the container with id 0681db6835f226d32127ace2627dc5eee121af810dce9c7f46fef8da54509cee
Sep 17 14:06:05.822110 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:05.822072    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c7fc9df5d-jl62r" event={"ID":"81211573-af3a-4b68-ac05-92fdbb7cb8b2","Type":"ContainerStarted","Data":"6e30c1240602b0f15d6c029b67af55eb4dbdc27e212231d43128eec7ccffb34d"}
Sep 17 14:06:05.822517 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:05.822128    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c7fc9df5d-jl62r" event={"ID":"81211573-af3a-4b68-ac05-92fdbb7cb8b2","Type":"ContainerStarted","Data":"df8e453457a8d9a1dd5a74ea035ca5277fcd74ea2a70972d4a3a5e372c402366"}
Sep 17 14:06:05.822517 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:05.822253    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-c7fc9df5d-jl62r"
Sep 17 14:06:05.823503 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:05.823482    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dg4gz" event={"ID":"a2cbe70b-eb96-4779-9143-b95a1204aae2","Type":"ContainerStarted","Data":"e183f9a12906ccbab4f223e0e8b75c8cef32e6e8123c0b520d3977b5924b723b"}
Sep 17 14:06:05.823633 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:05.823516    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dg4gz" event={"ID":"a2cbe70b-eb96-4779-9143-b95a1204aae2","Type":"ContainerStarted","Data":"d07c393b74c44d7820ccc6193e3a9a44b55b286adeee050ca41401414eab6a61"}
Sep 17 14:06:05.823633 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:05.823586    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-dg4gz"
Sep 17 14:06:05.824721 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:05.824690    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7k2wk" event={"ID":"31e72879-bbfc-4684-bcbb-fd79b4eec4b6","Type":"ContainerStarted","Data":"3f06098386e3a8ba7382928778dde57ae6c80041c801470825b2abe0c9cca414"}
Sep 17 14:06:05.825847 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:05.825827    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6v652" event={"ID":"40b77f59-aa77-4958-bd61-2522680f1ecf","Type":"ContainerStarted","Data":"4a0bd0ac8e8a89cf6dbbb4b5f14adc8b66393a2b944edb3041ae98518c679107"}
Sep 17 14:06:05.826874 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:05.826857    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-r4p4g" event={"ID":"059afb43-5e5e-4db9-bfa2-d874291c1c3a","Type":"ContainerStarted","Data":"0681db6835f226d32127ace2627dc5eee121af810dce9c7f46fef8da54509cee"}
Sep 17 14:06:05.844747 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:05.844703    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-c7fc9df5d-jl62r" podStartSLOduration=45.844689576 podStartE2EDuration="45.844689576s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:05:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:06:05.842352676 +0000 UTC m=+66.843896149" watchObservedRunningTime="2026-09-17 14:06:05.844689576 +0000 UTC m=+66.846233050"
Sep 17 14:06:05.858692 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:05.858555    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-dg4gz" podStartSLOduration=65.858535262 podStartE2EDuration="1m5.858535262s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:05:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:06:05.858421689 +0000 UTC m=+66.859965164" watchObservedRunningTime="2026-09-17 14:06:05.858535262 +0000 UTC m=+66.860078735"
Sep 17 14:06:06.206739 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:06.206702    2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-rr97k"
Sep 17 14:06:06.206913 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:06.206767    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-rr97k"
Sep 17 14:06:06.207137 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:06.207119    2582 scope.go:117] "RemoveContainer" containerID="38cac992b3d877cea1215d81be03306d6b9879d7b116edc03daae89cbdd280de"
Sep 17 14:06:06.207381 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:06:06.207356    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-696884fb9c-rr97k_openshift-console-operator(45ffa13d-635a-46e8-8894-30902b9cf719)\"" pod="openshift-console-operator/console-operator-696884fb9c-rr97k" podUID="45ffa13d-635a-46e8-8894-30902b9cf719"
Sep 17 14:06:07.747478 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:07.747442    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1414b9c2-7c96-4383-9947-e7abcd68a074-metrics-certs\") pod \"router-default-68854cb984-hm4cb\" (UID: \"1414b9c2-7c96-4383-9947-e7abcd68a074\") " pod="openshift-ingress/router-default-68854cb984-hm4cb"
Sep 17 14:06:07.747908 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:07.747498    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1414b9c2-7c96-4383-9947-e7abcd68a074-service-ca-bundle\") pod \"router-default-68854cb984-hm4cb\" (UID: \"1414b9c2-7c96-4383-9947-e7abcd68a074\") " pod="openshift-ingress/router-default-68854cb984-hm4cb"
Sep 17 14:06:07.748136 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:07.748116    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1414b9c2-7c96-4383-9947-e7abcd68a074-service-ca-bundle\") pod \"router-default-68854cb984-hm4cb\" (UID: \"1414b9c2-7c96-4383-9947-e7abcd68a074\") " pod="openshift-ingress/router-default-68854cb984-hm4cb"
Sep 17 14:06:07.749861 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:07.749829    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1414b9c2-7c96-4383-9947-e7abcd68a074-metrics-certs\") pod \"router-default-68854cb984-hm4cb\" (UID: \"1414b9c2-7c96-4383-9947-e7abcd68a074\") " pod="openshift-ingress/router-default-68854cb984-hm4cb"
Sep 17 14:06:07.838409 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:07.838315    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6v652" event={"ID":"40b77f59-aa77-4958-bd61-2522680f1ecf","Type":"ContainerStarted","Data":"33e0ed40121f8cc2588febbe5ec3bfd423e5b99085b114f87e0713798531b429"}
Sep 17 14:06:07.838409 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:07.838388    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6v652" event={"ID":"40b77f59-aa77-4958-bd61-2522680f1ecf","Type":"ContainerStarted","Data":"0e8c55c14e0a0604b66bee5143f812da572d763d10180431e565a0bb0e418d12"}
Sep 17 14:06:07.838587 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:07.838421    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-6v652"
Sep 17 14:06:07.840364 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:07.840335    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-r4p4g" event={"ID":"059afb43-5e5e-4db9-bfa2-d874291c1c3a","Type":"ContainerStarted","Data":"4e5e17f05fd4e66b45c632eed2c01c9d39b9fc9b83c0c9904c20e826024d9661"}
Sep 17 14:06:07.840494 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:07.840386    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-r4p4g" event={"ID":"059afb43-5e5e-4db9-bfa2-d874291c1c3a","Type":"ContainerStarted","Data":"bd492401c96f0bd2c945024b223b8a1aa9a640926533bca618addef6790d23b0"}
Sep 17 14:06:07.848243 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:07.848217    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-r58pd\" (UID: \"1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-r58pd"
Sep 17 14:06:07.850928 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:07.850897    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-r58pd\" (UID: \"1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-r58pd"
Sep 17 14:06:07.867533 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:07.867474    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-6v652" podStartSLOduration=33.828597795 podStartE2EDuration="35.867449412s" totalImagesPullingTime="2.038851617s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:05:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:06:07.866733385 +0000 UTC m=+68.868276869" watchObservedRunningTime="2026-09-17 14:06:07.867449412 +0000 UTC m=+68.868992886"
Sep 17 14:06:07.890105 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:07.890043    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-r4p4g" podStartSLOduration=66.270419657 podStartE2EDuration="1m7.890017462s" totalImagesPullingTime="1.619597805s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:05:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:06:07.889290773 +0000 UTC m=+68.890834247" watchObservedRunningTime="2026-09-17 14:06:07.890017462 +0000 UTC m=+68.891560940"
Sep 17 14:06:07.907360 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:07.907326    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-4r2cr\""
Sep 17 14:06:07.915403 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:07.915370    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-68854cb984-hm4cb"
Sep 17 14:06:07.949272 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:07.949232    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9fbd6f7b-2139-4134-9577-fce124537c73-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-47zgn\" (UID: \"9fbd6f7b-2139-4134-9577-fce124537c73\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-47zgn"
Sep 17 14:06:07.951966 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:07.951941    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9fbd6f7b-2139-4134-9577-fce124537c73-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-47zgn\" (UID: \"9fbd6f7b-2139-4134-9577-fce124537c73\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-47zgn"
Sep 17 14:06:08.023934 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:08.023901    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-7tbnk\""
Sep 17 14:06:08.031897 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:08.031870    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-r58pd"
Sep 17 14:06:08.050442 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:08.050408    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b9b37579-a215-4a77-bd0a-f33675b1b17f-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-7hhfj\" (UID: \"b9b37579-a215-4a77-bd0a-f33675b1b17f\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-7hhfj"
Sep 17 14:06:08.053063 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:08.053039    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b9b37579-a215-4a77-bd0a-f33675b1b17f-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-7hhfj\" (UID: \"b9b37579-a215-4a77-bd0a-f33675b1b17f\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-7hhfj"
Sep 17 14:06:08.122938 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:08.122909    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-jbr7g\""
Sep 17 14:06:08.130174 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:08.130153    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-47zgn"
Sep 17 14:06:08.198587 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:08.198559    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-68854cb984-hm4cb"]
Sep 17 14:06:08.199109 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:08.199002    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-9b46m\""
Sep 17 14:06:08.201700 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:06:08.201657    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1414b9c2_7c96_4383_9947_e7abcd68a074.slice/crio-8912f1930857b6d19d92e3dbe87d6bcbb14503caa71fcf9e96d025c38a3c9e55 WatchSource:0}: Error finding container 8912f1930857b6d19d92e3dbe87d6bcbb14503caa71fcf9e96d025c38a3c9e55: Status 404 returned error can't find the container with id 8912f1930857b6d19d92e3dbe87d6bcbb14503caa71fcf9e96d025c38a3c9e55
Sep 17 14:06:08.208196 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:08.208171    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-7hhfj"
Sep 17 14:06:08.322477 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:08.322300    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-r58pd"]
Sep 17 14:06:08.324820 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:06:08.324792    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c6950fd_ebfd_4e8e_acbe_6c6e72c00b99.slice/crio-ad8cb0c4038c0be209039ba351f6885f1f62f3fd6b31e4bd46048c64d77f75f5 WatchSource:0}: Error finding container ad8cb0c4038c0be209039ba351f6885f1f62f3fd6b31e4bd46048c64d77f75f5: Status 404 returned error can't find the container with id ad8cb0c4038c0be209039ba351f6885f1f62f3fd6b31e4bd46048c64d77f75f5
Sep 17 14:06:08.425700 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:08.425553    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-47zgn"]
Sep 17 14:06:08.435862 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:08.435829    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:08.498645 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:08.498599    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-7hhfj"]
Sep 17 14:06:08.502085 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:06:08.502059    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9b37579_a215_4a77_bd0a_f33675b1b17f.slice/crio-00bde7a124fddf83fa74d912192a01b1c21ce789cf177c63813c8e830d33680c WatchSource:0}: Error finding container 00bde7a124fddf83fa74d912192a01b1c21ce789cf177c63813c8e830d33680c: Status 404 returned error can't find the container with id 00bde7a124fddf83fa74d912192a01b1c21ce789cf177c63813c8e830d33680c
Sep 17 14:06:08.564810 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:08.564782    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:08.846502 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:08.846466    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-7hhfj" event={"ID":"b9b37579-a215-4a77-bd0a-f33675b1b17f","Type":"ContainerStarted","Data":"00bde7a124fddf83fa74d912192a01b1c21ce789cf177c63813c8e830d33680c"}
Sep 17 14:06:08.847961 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:08.847932    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-47zgn" event={"ID":"9fbd6f7b-2139-4134-9577-fce124537c73","Type":"ContainerStarted","Data":"27bb49c0c562e7db098233cec04e13ea07b74c4d3ec03eec4ec06e624b6a6b46"}
Sep 17 14:06:08.849299 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:08.849270    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-r58pd" event={"ID":"1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99","Type":"ContainerStarted","Data":"ad8cb0c4038c0be209039ba351f6885f1f62f3fd6b31e4bd46048c64d77f75f5"}
Sep 17 14:06:08.851071 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:08.850997    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-68854cb984-hm4cb" event={"ID":"1414b9c2-7c96-4383-9947-e7abcd68a074","Type":"ContainerStarted","Data":"01ea560ed559e305f9838b9eccc8494a367c412f2778d5c8fbb70298f17adb60"}
Sep 17 14:06:08.851071 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:08.851038    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-68854cb984-hm4cb" event={"ID":"1414b9c2-7c96-4383-9947-e7abcd68a074","Type":"ContainerStarted","Data":"8912f1930857b6d19d92e3dbe87d6bcbb14503caa71fcf9e96d025c38a3c9e55"}
Sep 17 14:06:08.876890 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:08.876823    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-68854cb984-hm4cb" podStartSLOduration=33.8768011 podStartE2EDuration="33.8768011s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:05:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:06:08.876620397 +0000 UTC m=+69.878163879" watchObservedRunningTime="2026-09-17 14:06:08.8768011 +0000 UTC m=+69.878344574"
Sep 17 14:06:08.915944 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:08.915902    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-68854cb984-hm4cb"
Sep 17 14:06:08.919018 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:08.918994    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-68854cb984-hm4cb"
Sep 17 14:06:09.854876 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:09.854848    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-68854cb984-hm4cb"
Sep 17 14:06:09.856323 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:09.856294    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-68854cb984-hm4cb"
Sep 17 14:06:10.865650 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:10.865544    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7k2wk" event={"ID":"31e72879-bbfc-4684-bcbb-fd79b4eec4b6","Type":"ContainerStarted","Data":"a081967f9ccd9ab0e6a1b9d97cab6a05b4eefd38e70700a9a2fd6a426cf0e95c"}
Sep 17 14:06:11.112743 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:11.112638    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:11.417342 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:11.417299    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:11.689151 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:11.689073    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:12.092743 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:12.092714    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:13.093733 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:13.093709    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:13.318436 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:13.318342    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:13.498264 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:13.498227    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:13.674953 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:13.674856    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-7k2wk" podStartSLOduration=68.795962264 podStartE2EDuration="1m13.674839403s" totalImagesPullingTime="4.878877139s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 14:05:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:06:10.937525277 +0000 UTC m=+71.939068752" watchObservedRunningTime="2026-09-17 14:06:13.674839403 +0000 UTC m=+74.676382876"
Sep 17 14:06:13.675406 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:13.675385    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-9ppw7"]
Sep 17 14:06:13.698984 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:13.698959    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-9ppw7"]
Sep 17 14:06:13.699117 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:13.699044    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-9ppw7"
Sep 17 14:06:13.701957 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:13.701937    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 17 14:06:13.702966 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:13.702939    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-8wpdd\""
Sep 17 14:06:13.802619 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:13.802585    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/8574e419-b1dd-4888-a942-f7c38df4795d-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-9ppw7\" (UID: \"8574e419-b1dd-4888-a942-f7c38df4795d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-9ppw7"
Sep 17 14:06:13.837547 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:13.837518    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:13.877906 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:13.877877    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-7hhfj" event={"ID":"b9b37579-a215-4a77-bd0a-f33675b1b17f","Type":"ContainerStarted","Data":"e285ead3413e0d6f3ec5c4b2a70160657a8931097419a351a0cc25e005b06381"}
Sep 17 14:06:13.879714 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:13.879689    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-47zgn" event={"ID":"9fbd6f7b-2139-4134-9577-fce124537c73","Type":"ContainerStarted","Data":"3d9413e9213a6f1140bf45872b9a65579b4e6a7160590280dd6b500bc35806b3"}
Sep 17 14:06:13.879854 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:13.879737    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-47zgn" event={"ID":"9fbd6f7b-2139-4134-9577-fce124537c73","Type":"ContainerStarted","Data":"e0ef7358de892781d5e9c2e254a3455ed8c3e2fffe66202c041a601bfc2cf412"}
Sep 17 14:06:13.881254 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:13.881231    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-r58pd" event={"ID":"1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99","Type":"ContainerStarted","Data":"ab0c917cdab99c5756c828a7c023388a3ce89ddc185a378b18faf0849d613d98"}
Sep 17 14:06:13.903831 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:13.903811    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/8574e419-b1dd-4888-a942-f7c38df4795d-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-9ppw7\" (UID: \"8574e419-b1dd-4888-a942-f7c38df4795d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-9ppw7"
Sep 17 14:06:13.903956 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:06:13.903942    2582 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Sep 17 14:06:13.904006 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:06:13.903996    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8574e419-b1dd-4888-a942-f7c38df4795d-tls-certificates podName:8574e419-b1dd-4888-a942-f7c38df4795d nodeName:}" failed. No retries permitted until 2026-09-17 14:06:14.403982101 +0000 UTC m=+75.405525557 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/8574e419-b1dd-4888-a942-f7c38df4795d-tls-certificates") pod "prometheus-operator-admission-webhook-5864cd99d7-9ppw7" (UID: "8574e419-b1dd-4888-a942-f7c38df4795d") : secret "prometheus-operator-admission-webhook-tls" not found
Sep 17 14:06:13.908197 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:13.908158    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-7hhfj" podStartSLOduration=33.403188706 podStartE2EDuration="37.908145392s" totalImagesPullingTime="4.504956686s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:05:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:06:13.905558121 +0000 UTC m=+74.907101594" watchObservedRunningTime="2026-09-17 14:06:13.908145392 +0000 UTC m=+74.909688866"
Sep 17 14:06:13.936448 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:13.936368    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-47zgn" podStartSLOduration=34.405738339 podStartE2EDuration="38.936353132s" totalImagesPullingTime="4.530614793s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:05:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:06:13.935033559 +0000 UTC m=+74.936577033" watchObservedRunningTime="2026-09-17 14:06:13.936353132 +0000 UTC m=+74.937896606"
Sep 17 14:06:13.966803 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:13.966752    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-r58pd" podStartSLOduration=34.285501886 podStartE2EDuration="38.966736428s" totalImagesPullingTime="4.681234542s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:05:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:06:13.965926644 +0000 UTC m=+74.967470118" watchObservedRunningTime="2026-09-17 14:06:13.966736428 +0000 UTC m=+74.968279902"
Sep 17 14:06:14.013473 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:14.013444    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:14.408296 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:14.408254    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/8574e419-b1dd-4888-a942-f7c38df4795d-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-9ppw7\" (UID: \"8574e419-b1dd-4888-a942-f7c38df4795d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-9ppw7"
Sep 17 14:06:14.408693 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:06:14.408372    2582 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Sep 17 14:06:14.408693 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:06:14.408426    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8574e419-b1dd-4888-a942-f7c38df4795d-tls-certificates podName:8574e419-b1dd-4888-a942-f7c38df4795d nodeName:}" failed. No retries permitted until 2026-09-17 14:06:15.408412759 +0000 UTC m=+76.409956225 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/8574e419-b1dd-4888-a942-f7c38df4795d-tls-certificates") pod "prometheus-operator-admission-webhook-5864cd99d7-9ppw7" (UID: "8574e419-b1dd-4888-a942-f7c38df4795d") : secret "prometheus-operator-admission-webhook-tls" not found
Sep 17 14:06:14.792274 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:14.792217    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:14.949317 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:14.949283    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:15.250899 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:15.250861    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:15.415235 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:15.415197    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/8574e419-b1dd-4888-a942-f7c38df4795d-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-9ppw7\" (UID: \"8574e419-b1dd-4888-a942-f7c38df4795d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-9ppw7"
Sep 17 14:06:15.417591 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:15.417566    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/8574e419-b1dd-4888-a942-f7c38df4795d-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-9ppw7\" (UID: \"8574e419-b1dd-4888-a942-f7c38df4795d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-9ppw7"
Sep 17 14:06:15.439498 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:15.439458    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:15.507488 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:15.507433    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-9ppw7"
Sep 17 14:06:15.832729 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:15.832705    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-9ppw7"]
Sep 17 14:06:15.835019 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:06:15.834990    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8574e419_b1dd_4888_a942_f7c38df4795d.slice/crio-4fdfdf2bbb18ab99f555d332f2b1ea7dd7c1684de2e02f368821378497d209e5 WatchSource:0}: Error finding container 4fdfdf2bbb18ab99f555d332f2b1ea7dd7c1684de2e02f368821378497d209e5: Status 404 returned error can't find the container with id 4fdfdf2bbb18ab99f555d332f2b1ea7dd7c1684de2e02f368821378497d209e5
Sep 17 14:06:15.890134 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:15.890106    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-9ppw7" event={"ID":"8574e419-b1dd-4888-a942-f7c38df4795d","Type":"ContainerStarted","Data":"4fdfdf2bbb18ab99f555d332f2b1ea7dd7c1684de2e02f368821378497d209e5"}
Sep 17 14:06:17.853689 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:17.853659    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-6v652"
Sep 17 14:06:17.900738 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:17.900703    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-9ppw7" event={"ID":"8574e419-b1dd-4888-a942-f7c38df4795d","Type":"ContainerStarted","Data":"68d066da5b3c2dbfa79466f4e041317dc502c7931656b473704fa068d81370c4"}
Sep 17 14:06:17.900901 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:17.900873    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-9ppw7"
Sep 17 14:06:17.905363 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:17.905340    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-9ppw7"
Sep 17 14:06:17.922366 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:17.922320    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-9ppw7" podStartSLOduration=3.7800365510000002 podStartE2EDuration="4.922305009s" totalImagesPullingTime="1.142268458s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:06:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:06:17.921597779 +0000 UTC m=+78.923141253" watchObservedRunningTime="2026-09-17 14:06:17.922305009 +0000 UTC m=+78.923848483"
Sep 17 14:06:18.803931 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:18.803893    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:19.016510 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:19.016468    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:19.488433 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:19.488403    2582 scope.go:117] "RemoveContainer" containerID="38cac992b3d877cea1215d81be03306d6b9879d7b116edc03daae89cbdd280de"
Sep 17 14:06:19.488645 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:06:19.488598    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-696884fb9c-rr97k_openshift-console-operator(45ffa13d-635a-46e8-8894-30902b9cf719)\"" pod="openshift-console-operator/console-operator-696884fb9c-rr97k" podUID="45ffa13d-635a-46e8-8894-30902b9cf719"
Sep 17 14:06:21.070464 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:21.070420    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:22.989847 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:22.989663    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:23.193937 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.193904    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:23.225120 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.225085    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-6hsrq"]
Sep 17 14:06:23.227378 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.227363    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6hsrq"
Sep 17 14:06:23.229848 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.229826    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-t8ndb\""
Sep 17 14:06:23.231491 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.231464    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 17 14:06:23.231652 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.231633    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 17 14:06:23.231847 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.231826    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 17 14:06:23.252283 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.252223    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-6hsrq"]
Sep 17 14:06:23.313691 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.313665    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-vh2mf"]
Sep 17 14:06:23.315784 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.315769    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-5qnzn"]
Sep 17 14:06:23.315908 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.315890    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vh2mf"
Sep 17 14:06:23.317876 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.317854    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-5qnzn"
Sep 17 14:06:23.321706 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.321690    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5jnvg\""
Sep 17 14:06:23.321804 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.321723    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 17 14:06:23.321896 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.321884    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 17 14:06:23.322645 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.322625    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 17 14:06:23.335986 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.335961    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-rqjsk\""
Sep 17 14:06:23.336181 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.336164    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 17 14:06:23.336263 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.336197    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 17 14:06:23.336263 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.336254    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 17 14:06:23.370704 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.370682    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-5qnzn"]
Sep 17 14:06:23.379170 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.379135    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fc1de48c-04a5-4a21-be57-66a4e91b3947-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-6hsrq\" (UID: \"fc1de48c-04a5-4a21-be57-66a4e91b3947\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6hsrq"
Sep 17 14:06:23.379268 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.379185    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9k5d2\" (UniqueName: \"kubernetes.io/projected/fc1de48c-04a5-4a21-be57-66a4e91b3947-kube-api-access-9k5d2\") pod \"openshift-state-metrics-67dcf8c979-6hsrq\" (UID: \"fc1de48c-04a5-4a21-be57-66a4e91b3947\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6hsrq"
Sep 17 14:06:23.379268 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.379204    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fc1de48c-04a5-4a21-be57-66a4e91b3947-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-6hsrq\" (UID: \"fc1de48c-04a5-4a21-be57-66a4e91b3947\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6hsrq"
Sep 17 14:06:23.379353 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.379302    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fc1de48c-04a5-4a21-be57-66a4e91b3947-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-6hsrq\" (UID: \"fc1de48c-04a5-4a21-be57-66a4e91b3947\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6hsrq"
Sep 17 14:06:23.479863 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.479833    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c187d7ca-b3c3-4fe0-9e5c-a2c9fc28c347-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-5qnzn\" (UID: \"c187d7ca-b3c3-4fe0-9e5c-a2c9fc28c347\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-5qnzn"
Sep 17 14:06:23.479863 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.479869    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c187d7ca-b3c3-4fe0-9e5c-a2c9fc28c347-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-5qnzn\" (UID: \"c187d7ca-b3c3-4fe0-9e5c-a2c9fc28c347\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-5qnzn"
Sep 17 14:06:23.480068 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.479892    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/357b3c15-b702-46dd-949f-d9c2fcd8872d-root\") pod \"node-exporter-vh2mf\" (UID: \"357b3c15-b702-46dd-949f-d9c2fcd8872d\") " pod="openshift-monitoring/node-exporter-vh2mf"
Sep 17 14:06:23.480068 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.479909    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c187d7ca-b3c3-4fe0-9e5c-a2c9fc28c347-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-5qnzn\" (UID: \"c187d7ca-b3c3-4fe0-9e5c-a2c9fc28c347\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-5qnzn"
Sep 17 14:06:23.480068 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.479973    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggjzg\" (UniqueName: \"kubernetes.io/projected/357b3c15-b702-46dd-949f-d9c2fcd8872d-kube-api-access-ggjzg\") pod \"node-exporter-vh2mf\" (UID: \"357b3c15-b702-46dd-949f-d9c2fcd8872d\") " pod="openshift-monitoring/node-exporter-vh2mf"
Sep 17 14:06:23.480068 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.480032    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fc1de48c-04a5-4a21-be57-66a4e91b3947-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-6hsrq\" (UID: \"fc1de48c-04a5-4a21-be57-66a4e91b3947\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6hsrq"
Sep 17 14:06:23.480068 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.480064    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/357b3c15-b702-46dd-949f-d9c2fcd8872d-node-exporter-wtmp\") pod \"node-exporter-vh2mf\" (UID: \"357b3c15-b702-46dd-949f-d9c2fcd8872d\") " pod="openshift-monitoring/node-exporter-vh2mf"
Sep 17 14:06:23.480237 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.480085    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/357b3c15-b702-46dd-949f-d9c2fcd8872d-metrics-client-ca\") pod \"node-exporter-vh2mf\" (UID: \"357b3c15-b702-46dd-949f-d9c2fcd8872d\") " pod="openshift-monitoring/node-exporter-vh2mf"
Sep 17 14:06:23.480237 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.480104    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c187d7ca-b3c3-4fe0-9e5c-a2c9fc28c347-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-5qnzn\" (UID: \"c187d7ca-b3c3-4fe0-9e5c-a2c9fc28c347\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-5qnzn"
Sep 17 14:06:23.480237 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.480223    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fc1de48c-04a5-4a21-be57-66a4e91b3947-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-6hsrq\" (UID: \"fc1de48c-04a5-4a21-be57-66a4e91b3947\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6hsrq"
Sep 17 14:06:23.480336 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.480241    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/357b3c15-b702-46dd-949f-d9c2fcd8872d-node-exporter-accelerators-collector-config\") pod \"node-exporter-vh2mf\" (UID: \"357b3c15-b702-46dd-949f-d9c2fcd8872d\") " pod="openshift-monitoring/node-exporter-vh2mf"
Sep 17 14:06:23.480336 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.480261    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/357b3c15-b702-46dd-949f-d9c2fcd8872d-node-exporter-tls\") pod \"node-exporter-vh2mf\" (UID: \"357b3c15-b702-46dd-949f-d9c2fcd8872d\") " pod="openshift-monitoring/node-exporter-vh2mf"
Sep 17 14:06:23.480336 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.480277    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9k5d2\" (UniqueName: \"kubernetes.io/projected/fc1de48c-04a5-4a21-be57-66a4e91b3947-kube-api-access-9k5d2\") pod \"openshift-state-metrics-67dcf8c979-6hsrq\" (UID: \"fc1de48c-04a5-4a21-be57-66a4e91b3947\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6hsrq"
Sep 17 14:06:23.480336 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.480300    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fc1de48c-04a5-4a21-be57-66a4e91b3947-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-6hsrq\" (UID: \"fc1de48c-04a5-4a21-be57-66a4e91b3947\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6hsrq"
Sep 17 14:06:23.480336 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.480325    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/357b3c15-b702-46dd-949f-d9c2fcd8872d-sys\") pod \"node-exporter-vh2mf\" (UID: \"357b3c15-b702-46dd-949f-d9c2fcd8872d\") " pod="openshift-monitoring/node-exporter-vh2mf"
Sep 17 14:06:23.480528 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.480343    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/357b3c15-b702-46dd-949f-d9c2fcd8872d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vh2mf\" (UID: \"357b3c15-b702-46dd-949f-d9c2fcd8872d\") " pod="openshift-monitoring/node-exporter-vh2mf"
Sep 17 14:06:23.480528 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.480360    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tgc4\" (UniqueName: \"kubernetes.io/projected/c187d7ca-b3c3-4fe0-9e5c-a2c9fc28c347-kube-api-access-2tgc4\") pod \"kube-state-metrics-59fbdd77d-5qnzn\" (UID: \"c187d7ca-b3c3-4fe0-9e5c-a2c9fc28c347\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-5qnzn"
Sep 17 14:06:23.480627 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.480556    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/357b3c15-b702-46dd-949f-d9c2fcd8872d-node-exporter-textfile\") pod \"node-exporter-vh2mf\" (UID: \"357b3c15-b702-46dd-949f-d9c2fcd8872d\") " pod="openshift-monitoring/node-exporter-vh2mf"
Sep 17 14:06:23.480627 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.480582    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c187d7ca-b3c3-4fe0-9e5c-a2c9fc28c347-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-5qnzn\" (UID: \"c187d7ca-b3c3-4fe0-9e5c-a2c9fc28c347\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-5qnzn"
Sep 17 14:06:23.481105 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.481078    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fc1de48c-04a5-4a21-be57-66a4e91b3947-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-6hsrq\" (UID: \"fc1de48c-04a5-4a21-be57-66a4e91b3947\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6hsrq"
Sep 17 14:06:23.482833 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.482808    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fc1de48c-04a5-4a21-be57-66a4e91b3947-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-6hsrq\" (UID: \"fc1de48c-04a5-4a21-be57-66a4e91b3947\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6hsrq"
Sep 17 14:06:23.482833 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.482827    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fc1de48c-04a5-4a21-be57-66a4e91b3947-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-6hsrq\" (UID: \"fc1de48c-04a5-4a21-be57-66a4e91b3947\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6hsrq"
Sep 17 14:06:23.497728 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.497703    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9k5d2\" (UniqueName: \"kubernetes.io/projected/fc1de48c-04a5-4a21-be57-66a4e91b3947-kube-api-access-9k5d2\") pod \"openshift-state-metrics-67dcf8c979-6hsrq\" (UID: \"fc1de48c-04a5-4a21-be57-66a4e91b3947\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6hsrq"
Sep 17 14:06:23.536718 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.536652    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6hsrq"
Sep 17 14:06:23.581714 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.581671    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/357b3c15-b702-46dd-949f-d9c2fcd8872d-node-exporter-wtmp\") pod \"node-exporter-vh2mf\" (UID: \"357b3c15-b702-46dd-949f-d9c2fcd8872d\") " pod="openshift-monitoring/node-exporter-vh2mf"
Sep 17 14:06:23.581837 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.581726    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/357b3c15-b702-46dd-949f-d9c2fcd8872d-metrics-client-ca\") pod \"node-exporter-vh2mf\" (UID: \"357b3c15-b702-46dd-949f-d9c2fcd8872d\") " pod="openshift-monitoring/node-exporter-vh2mf"
Sep 17 14:06:23.581837 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.581758    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c187d7ca-b3c3-4fe0-9e5c-a2c9fc28c347-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-5qnzn\" (UID: \"c187d7ca-b3c3-4fe0-9e5c-a2c9fc28c347\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-5qnzn"
Sep 17 14:06:23.581837 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.581803    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/357b3c15-b702-46dd-949f-d9c2fcd8872d-node-exporter-accelerators-collector-config\") pod \"node-exporter-vh2mf\" (UID: \"357b3c15-b702-46dd-949f-d9c2fcd8872d\") " pod="openshift-monitoring/node-exporter-vh2mf"
Sep 17 14:06:23.582019 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.581839    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/357b3c15-b702-46dd-949f-d9c2fcd8872d-node-exporter-tls\") pod \"node-exporter-vh2mf\" (UID: \"357b3c15-b702-46dd-949f-d9c2fcd8872d\") " pod="openshift-monitoring/node-exporter-vh2mf"
Sep 17 14:06:23.582019 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.581871    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/357b3c15-b702-46dd-949f-d9c2fcd8872d-sys\") pod \"node-exporter-vh2mf\" (UID: \"357b3c15-b702-46dd-949f-d9c2fcd8872d\") " pod="openshift-monitoring/node-exporter-vh2mf"
Sep 17 14:06:23.582019 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.581899    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/357b3c15-b702-46dd-949f-d9c2fcd8872d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vh2mf\" (UID: \"357b3c15-b702-46dd-949f-d9c2fcd8872d\") " pod="openshift-monitoring/node-exporter-vh2mf"
Sep 17 14:06:23.582019 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.581927    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tgc4\" (UniqueName: \"kubernetes.io/projected/c187d7ca-b3c3-4fe0-9e5c-a2c9fc28c347-kube-api-access-2tgc4\") pod \"kube-state-metrics-59fbdd77d-5qnzn\" (UID: \"c187d7ca-b3c3-4fe0-9e5c-a2c9fc28c347\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-5qnzn"
Sep 17 14:06:23.582019 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.581971    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/357b3c15-b702-46dd-949f-d9c2fcd8872d-node-exporter-textfile\") pod \"node-exporter-vh2mf\" (UID: \"357b3c15-b702-46dd-949f-d9c2fcd8872d\") " pod="openshift-monitoring/node-exporter-vh2mf"
Sep 17 14:06:23.582019 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.581992    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c187d7ca-b3c3-4fe0-9e5c-a2c9fc28c347-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-5qnzn\" (UID: \"c187d7ca-b3c3-4fe0-9e5c-a2c9fc28c347\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-5qnzn"
Sep 17 14:06:23.582341 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.582022    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c187d7ca-b3c3-4fe0-9e5c-a2c9fc28c347-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-5qnzn\" (UID: \"c187d7ca-b3c3-4fe0-9e5c-a2c9fc28c347\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-5qnzn"
Sep 17 14:06:23.582341 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.582059    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c187d7ca-b3c3-4fe0-9e5c-a2c9fc28c347-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-5qnzn\" (UID: \"c187d7ca-b3c3-4fe0-9e5c-a2c9fc28c347\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-5qnzn"
Sep 17 14:06:23.582341 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.582100    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/357b3c15-b702-46dd-949f-d9c2fcd8872d-root\") pod \"node-exporter-vh2mf\" (UID: \"357b3c15-b702-46dd-949f-d9c2fcd8872d\") " pod="openshift-monitoring/node-exporter-vh2mf"
Sep 17 14:06:23.582341 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.582142    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c187d7ca-b3c3-4fe0-9e5c-a2c9fc28c347-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-5qnzn\" (UID: \"c187d7ca-b3c3-4fe0-9e5c-a2c9fc28c347\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-5qnzn"
Sep 17 14:06:23.582341 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.582189    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggjzg\" (UniqueName: \"kubernetes.io/projected/357b3c15-b702-46dd-949f-d9c2fcd8872d-kube-api-access-ggjzg\") pod \"node-exporter-vh2mf\" (UID: \"357b3c15-b702-46dd-949f-d9c2fcd8872d\") " pod="openshift-monitoring/node-exporter-vh2mf"
Sep 17 14:06:23.583587 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.582975    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/357b3c15-b702-46dd-949f-d9c2fcd8872d-node-exporter-textfile\") pod \"node-exporter-vh2mf\" (UID: \"357b3c15-b702-46dd-949f-d9c2fcd8872d\") " pod="openshift-monitoring/node-exporter-vh2mf"
Sep 17 14:06:23.583587 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.583048    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/357b3c15-b702-46dd-949f-d9c2fcd8872d-root\") pod \"node-exporter-vh2mf\" (UID: \"357b3c15-b702-46dd-949f-d9c2fcd8872d\") " pod="openshift-monitoring/node-exporter-vh2mf"
Sep 17 14:06:23.583587 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.583089    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c187d7ca-b3c3-4fe0-9e5c-a2c9fc28c347-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-5qnzn\" (UID: \"c187d7ca-b3c3-4fe0-9e5c-a2c9fc28c347\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-5qnzn"
Sep 17 14:06:23.583587 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:06:23.583377    2582 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 17 14:06:23.583587 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:06:23.583448    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/357b3c15-b702-46dd-949f-d9c2fcd8872d-node-exporter-tls podName:357b3c15-b702-46dd-949f-d9c2fcd8872d nodeName:}" failed. No retries permitted until 2026-09-17 14:06:24.083427428 +0000 UTC m=+85.084970879 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/357b3c15-b702-46dd-949f-d9c2fcd8872d-node-exporter-tls") pod "node-exporter-vh2mf" (UID: "357b3c15-b702-46dd-949f-d9c2fcd8872d") : secret "node-exporter-tls" not found
Sep 17 14:06:23.584050 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.583590    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/357b3c15-b702-46dd-949f-d9c2fcd8872d-sys\") pod \"node-exporter-vh2mf\" (UID: \"357b3c15-b702-46dd-949f-d9c2fcd8872d\") " pod="openshift-monitoring/node-exporter-vh2mf"
Sep 17 14:06:23.584050 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.583754    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/357b3c15-b702-46dd-949f-d9c2fcd8872d-node-exporter-wtmp\") pod \"node-exporter-vh2mf\" (UID: \"357b3c15-b702-46dd-949f-d9c2fcd8872d\") " pod="openshift-monitoring/node-exporter-vh2mf"
Sep 17 14:06:23.584050 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.583844    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c187d7ca-b3c3-4fe0-9e5c-a2c9fc28c347-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-5qnzn\" (UID: \"c187d7ca-b3c3-4fe0-9e5c-a2c9fc28c347\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-5qnzn"
Sep 17 14:06:23.584050 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.583979    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c187d7ca-b3c3-4fe0-9e5c-a2c9fc28c347-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-5qnzn\" (UID: \"c187d7ca-b3c3-4fe0-9e5c-a2c9fc28c347\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-5qnzn"
Sep 17 14:06:23.587450 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.584364    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/357b3c15-b702-46dd-949f-d9c2fcd8872d-node-exporter-accelerators-collector-config\") pod \"node-exporter-vh2mf\" (UID: \"357b3c15-b702-46dd-949f-d9c2fcd8872d\") " pod="openshift-monitoring/node-exporter-vh2mf"
Sep 17 14:06:23.587450 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.584947    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/357b3c15-b702-46dd-949f-d9c2fcd8872d-metrics-client-ca\") pod \"node-exporter-vh2mf\" (UID: \"357b3c15-b702-46dd-949f-d9c2fcd8872d\") " pod="openshift-monitoring/node-exporter-vh2mf"
Sep 17 14:06:23.587450 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.586875    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/357b3c15-b702-46dd-949f-d9c2fcd8872d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vh2mf\" (UID: \"357b3c15-b702-46dd-949f-d9c2fcd8872d\") " pod="openshift-monitoring/node-exporter-vh2mf"
Sep 17 14:06:23.587658 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.587473    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c187d7ca-b3c3-4fe0-9e5c-a2c9fc28c347-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-5qnzn\" (UID: \"c187d7ca-b3c3-4fe0-9e5c-a2c9fc28c347\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-5qnzn"
Sep 17 14:06:23.587658 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.587532    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c187d7ca-b3c3-4fe0-9e5c-a2c9fc28c347-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-5qnzn\" (UID: \"c187d7ca-b3c3-4fe0-9e5c-a2c9fc28c347\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-5qnzn"
Sep 17 14:06:23.591407 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.591385    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:23.591551 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.591528    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggjzg\" (UniqueName: \"kubernetes.io/projected/357b3c15-b702-46dd-949f-d9c2fcd8872d-kube-api-access-ggjzg\") pod \"node-exporter-vh2mf\" (UID: \"357b3c15-b702-46dd-949f-d9c2fcd8872d\") " pod="openshift-monitoring/node-exporter-vh2mf"
Sep 17 14:06:23.594933 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.594908    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tgc4\" (UniqueName: \"kubernetes.io/projected/c187d7ca-b3c3-4fe0-9e5c-a2c9fc28c347-kube-api-access-2tgc4\") pod \"kube-state-metrics-59fbdd77d-5qnzn\" (UID: \"c187d7ca-b3c3-4fe0-9e5c-a2c9fc28c347\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-5qnzn"
Sep 17 14:06:23.631036 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.631009    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-5qnzn"
Sep 17 14:06:23.828562 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.828539    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-6hsrq"]
Sep 17 14:06:23.830446 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:06:23.830423    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc1de48c_04a5_4a21_be57_66a4e91b3947.slice/crio-b478b212246de07362b7495f5dc27a02b416dad61e9dc66bba0e8179769a1394 WatchSource:0}: Error finding container b478b212246de07362b7495f5dc27a02b416dad61e9dc66bba0e8179769a1394: Status 404 returned error can't find the container with id b478b212246de07362b7495f5dc27a02b416dad61e9dc66bba0e8179769a1394
Sep 17 14:06:23.920524 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.920457    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-5qnzn"]
Sep 17 14:06:23.923439 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.923415    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6hsrq" event={"ID":"fc1de48c-04a5-4a21-be57-66a4e91b3947","Type":"ContainerStarted","Data":"dc5ab4d3fcd08c5d9eb763f3af634e2b3f59c1629af3fa9c976954f3d8716689"}
Sep 17 14:06:23.923527 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.923461    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6hsrq" event={"ID":"fc1de48c-04a5-4a21-be57-66a4e91b3947","Type":"ContainerStarted","Data":"b478b212246de07362b7495f5dc27a02b416dad61e9dc66bba0e8179769a1394"}
Sep 17 14:06:23.929596 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:06:23.929567    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc187d7ca_b3c3_4fe0_9e5c_a2c9fc28c347.slice/crio-f4dbbe09a9f997824239c0c739d451bed5b4bd35d259a3d14939f93333c07fe7 WatchSource:0}: Error finding container f4dbbe09a9f997824239c0c739d451bed5b4bd35d259a3d14939f93333c07fe7: Status 404 returned error can't find the container with id f4dbbe09a9f997824239c0c739d451bed5b4bd35d259a3d14939f93333c07fe7
Sep 17 14:06:23.949835 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:23.949814    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:24.086992 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:24.086912    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/357b3c15-b702-46dd-949f-d9c2fcd8872d-node-exporter-tls\") pod \"node-exporter-vh2mf\" (UID: \"357b3c15-b702-46dd-949f-d9c2fcd8872d\") " pod="openshift-monitoring/node-exporter-vh2mf"
Sep 17 14:06:24.089179 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:24.089161    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/357b3c15-b702-46dd-949f-d9c2fcd8872d-node-exporter-tls\") pod \"node-exporter-vh2mf\" (UID: \"357b3c15-b702-46dd-949f-d9c2fcd8872d\") " pod="openshift-monitoring/node-exporter-vh2mf"
Sep 17 14:06:24.225392 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:24.225363    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vh2mf"
Sep 17 14:06:24.234864 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:06:24.234831    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod357b3c15_b702_46dd_949f_d9c2fcd8872d.slice/crio-e308dbcd4c302c1d5ea685fb9368dcf999bf1db2987430abe37e2a270464f784 WatchSource:0}: Error finding container e308dbcd4c302c1d5ea685fb9368dcf999bf1db2987430abe37e2a270464f784: Status 404 returned error can't find the container with id e308dbcd4c302c1d5ea685fb9368dcf999bf1db2987430abe37e2a270464f784
Sep 17 14:06:24.437298 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:24.437215    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:24.683779 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:24.683747    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:24.930583 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:24.930543    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-5qnzn" event={"ID":"c187d7ca-b3c3-4fe0-9e5c-a2c9fc28c347","Type":"ContainerStarted","Data":"f4dbbe09a9f997824239c0c739d451bed5b4bd35d259a3d14939f93333c07fe7"}
Sep 17 14:06:24.932987 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:24.932951    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6hsrq" event={"ID":"fc1de48c-04a5-4a21-be57-66a4e91b3947","Type":"ContainerStarted","Data":"ef4de9b6132d574db7a67d7721654e272dced72ed3856ea99d743445c90aece1"}
Sep 17 14:06:24.934542 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:24.934516    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vh2mf" event={"ID":"357b3c15-b702-46dd-949f-d9c2fcd8872d","Type":"ContainerStarted","Data":"e308dbcd4c302c1d5ea685fb9368dcf999bf1db2987430abe37e2a270464f784"}
Sep 17 14:06:25.006550 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:25.006504    2582 patch_prober.go:28] interesting pod/image-registry-c7fc9df5d-jl62r container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 17 14:06:25.006730 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:25.006575    2582 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-c7fc9df5d-jl62r" podUID="81211573-af3a-4b68-ac05-92fdbb7cb8b2" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 14:06:25.013098 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:25.013070    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:25.940730 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:25.940687    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6hsrq" event={"ID":"fc1de48c-04a5-4a21-be57-66a4e91b3947","Type":"ContainerStarted","Data":"f3160ba85a89b0d84725235f6474ad32475bf339f0e0f4c977b27bc035d6f17a"}
Sep 17 14:06:25.942549 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:25.942522    2582 generic.go:356] "Generic (PLEG): container finished" podID="357b3c15-b702-46dd-949f-d9c2fcd8872d" containerID="4b477ebf5c88a7b2a38227bacae86bad99abd11613deb426b74d251355654cee" exitCode=0
Sep 17 14:06:25.942851 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:25.942602    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vh2mf" event={"ID":"357b3c15-b702-46dd-949f-d9c2fcd8872d","Type":"ContainerDied","Data":"4b477ebf5c88a7b2a38227bacae86bad99abd11613deb426b74d251355654cee"}
Sep 17 14:06:25.945184 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:25.945149    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-5qnzn" event={"ID":"c187d7ca-b3c3-4fe0-9e5c-a2c9fc28c347","Type":"ContainerStarted","Data":"065f8c929ce9ff81dd422019a185a5e8f760694b0f961dce0b0eace9d4be458a"}
Sep 17 14:06:25.945350 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:25.945231    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-5qnzn" event={"ID":"c187d7ca-b3c3-4fe0-9e5c-a2c9fc28c347","Type":"ContainerStarted","Data":"b075940c57a304317787834cd86060c51138d0abc1cd6ba4c990fe73668397d7"}
Sep 17 14:06:25.945350 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:25.945263    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-5qnzn" event={"ID":"c187d7ca-b3c3-4fe0-9e5c-a2c9fc28c347","Type":"ContainerStarted","Data":"c1f19cf2c0ab9da939bb89ded9877c994a5a073fb3668498bf5ecfd4ea7d716b"}
Sep 17 14:06:25.976653 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:25.976571    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6hsrq" podStartSLOduration=1.362296156 podStartE2EDuration="2.976551245s" totalImagesPullingTime="1.614255089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:06:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:06:25.976277518 +0000 UTC m=+86.977820993" watchObservedRunningTime="2026-09-17 14:06:25.976551245 +0000 UTC m=+86.978094720"
Sep 17 14:06:26.022218 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:26.022171    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-5qnzn" podStartSLOduration=1.3903898049999999 podStartE2EDuration="3.022150103s" totalImagesPullingTime="1.631760298s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:06:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:06:26.020956587 +0000 UTC m=+87.022500064" watchObservedRunningTime="2026-09-17 14:06:26.022150103 +0000 UTC m=+87.023693576"
Sep 17 14:06:26.330099 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:26.330018    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:26.459301 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:26.459271    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:26.673902 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:26.673822    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:26.792620 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:26.792583    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:26.835909 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:26.835881    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-c7fc9df5d-jl62r"
Sep 17 14:06:26.954025 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:26.953989    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vh2mf" event={"ID":"357b3c15-b702-46dd-949f-d9c2fcd8872d","Type":"ContainerStarted","Data":"9d37b656234513f9780b64ed26d5f5078d2b95ddb05d1e41de0bb8e629f8f34b"}
Sep 17 14:06:26.954405 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:26.954051    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vh2mf" event={"ID":"357b3c15-b702-46dd-949f-d9c2fcd8872d","Type":"ContainerStarted","Data":"db87a78104e400e29c82f6f3f7f857a3674f38d3650815037e1b4858237d8973"}
Sep 17 14:06:26.979914 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:26.979871    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-vh2mf" podStartSLOduration=2.56921763 podStartE2EDuration="3.979854389s" totalImagesPullingTime="1.324109473s" totalInitContainerRuntime="86.527286ms" isStatefulPod=true podCreationTimestamp="2026-09-17 14:06:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:06:26.979326192 +0000 UTC m=+87.980869668" watchObservedRunningTime="2026-09-17 14:06:26.979854389 +0000 UTC m=+87.981397874"
Sep 17 14:06:27.239047 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:27.238956    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:27.410805 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:27.410775    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:27.790568 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:27.790539    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-67d9d56b4b-545xf"]
Sep 17 14:06:27.792702 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:27.792687    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-67d9d56b4b-545xf"
Sep 17 14:06:27.796464 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:27.796441    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 17 14:06:27.796649 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:27.796628    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-aihq5pvg54koe\""
Sep 17 14:06:27.796740 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:27.796665    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 17 14:06:27.796943 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:27.796927    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 17 14:06:27.797009 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:27.796946    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-bhxrv\""
Sep 17 14:06:27.797009 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:27.797005    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 17 14:06:27.812360 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:27.812332    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-67d9d56b4b-545xf"]
Sep 17 14:06:27.922412 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:27.922381    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/840db932-27b6-4ac5-8420-926b7277f4a3-secret-metrics-server-tls\") pod \"metrics-server-67d9d56b4b-545xf\" (UID: \"840db932-27b6-4ac5-8420-926b7277f4a3\") " pod="openshift-monitoring/metrics-server-67d9d56b4b-545xf"
Sep 17 14:06:27.922584 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:27.922428    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/840db932-27b6-4ac5-8420-926b7277f4a3-secret-metrics-server-client-certs\") pod \"metrics-server-67d9d56b4b-545xf\" (UID: \"840db932-27b6-4ac5-8420-926b7277f4a3\") " pod="openshift-monitoring/metrics-server-67d9d56b4b-545xf"
Sep 17 14:06:27.922584 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:27.922467    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/840db932-27b6-4ac5-8420-926b7277f4a3-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-67d9d56b4b-545xf\" (UID: \"840db932-27b6-4ac5-8420-926b7277f4a3\") " pod="openshift-monitoring/metrics-server-67d9d56b4b-545xf"
Sep 17 14:06:27.922584 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:27.922483    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xbqz\" (UniqueName: \"kubernetes.io/projected/840db932-27b6-4ac5-8420-926b7277f4a3-kube-api-access-5xbqz\") pod \"metrics-server-67d9d56b4b-545xf\" (UID: \"840db932-27b6-4ac5-8420-926b7277f4a3\") " pod="openshift-monitoring/metrics-server-67d9d56b4b-545xf"
Sep 17 14:06:27.922584 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:27.922502    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/840db932-27b6-4ac5-8420-926b7277f4a3-audit-log\") pod \"metrics-server-67d9d56b4b-545xf\" (UID: \"840db932-27b6-4ac5-8420-926b7277f4a3\") " pod="openshift-monitoring/metrics-server-67d9d56b4b-545xf"
Sep 17 14:06:27.922584 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:27.922518    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/840db932-27b6-4ac5-8420-926b7277f4a3-metrics-server-audit-profiles\") pod \"metrics-server-67d9d56b4b-545xf\" (UID: \"840db932-27b6-4ac5-8420-926b7277f4a3\") " pod="openshift-monitoring/metrics-server-67d9d56b4b-545xf"
Sep 17 14:06:27.922584 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:27.922558    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/840db932-27b6-4ac5-8420-926b7277f4a3-client-ca-bundle\") pod \"metrics-server-67d9d56b4b-545xf\" (UID: \"840db932-27b6-4ac5-8420-926b7277f4a3\") " pod="openshift-monitoring/metrics-server-67d9d56b4b-545xf"
Sep 17 14:06:27.943042 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:27.943009    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-tcmcl"]
Sep 17 14:06:27.945273 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:27.945253    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-tcmcl"
Sep 17 14:06:27.949045 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:27.949024    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 17 14:06:27.949152 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:27.949133    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-qhqxv\""
Sep 17 14:06:27.960715 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:27.960686    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-tcmcl"]
Sep 17 14:06:28.023780 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:28.023743    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/840db932-27b6-4ac5-8420-926b7277f4a3-secret-metrics-server-tls\") pod \"metrics-server-67d9d56b4b-545xf\" (UID: \"840db932-27b6-4ac5-8420-926b7277f4a3\") " pod="openshift-monitoring/metrics-server-67d9d56b4b-545xf"
Sep 17 14:06:28.023780 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:28.023787    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/840db932-27b6-4ac5-8420-926b7277f4a3-secret-metrics-server-client-certs\") pod \"metrics-server-67d9d56b4b-545xf\" (UID: \"840db932-27b6-4ac5-8420-926b7277f4a3\") " pod="openshift-monitoring/metrics-server-67d9d56b4b-545xf"
Sep 17 14:06:28.024028 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:28.023813    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/840db932-27b6-4ac5-8420-926b7277f4a3-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-67d9d56b4b-545xf\" (UID: \"840db932-27b6-4ac5-8420-926b7277f4a3\") " pod="openshift-monitoring/metrics-server-67d9d56b4b-545xf"
Sep 17 14:06:28.024028 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:28.023831    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xbqz\" (UniqueName: \"kubernetes.io/projected/840db932-27b6-4ac5-8420-926b7277f4a3-kube-api-access-5xbqz\") pod \"metrics-server-67d9d56b4b-545xf\" (UID: \"840db932-27b6-4ac5-8420-926b7277f4a3\") " pod="openshift-monitoring/metrics-server-67d9d56b4b-545xf"
Sep 17 14:06:28.024028 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:28.023854    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/840db932-27b6-4ac5-8420-926b7277f4a3-audit-log\") pod \"metrics-server-67d9d56b4b-545xf\" (UID: \"840db932-27b6-4ac5-8420-926b7277f4a3\") " pod="openshift-monitoring/metrics-server-67d9d56b4b-545xf"
Sep 17 14:06:28.024028 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:28.023877    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/840db932-27b6-4ac5-8420-926b7277f4a3-metrics-server-audit-profiles\") pod \"metrics-server-67d9d56b4b-545xf\" (UID: \"840db932-27b6-4ac5-8420-926b7277f4a3\") " pod="openshift-monitoring/metrics-server-67d9d56b4b-545xf"
Sep 17 14:06:28.024028 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:28.023923    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/840db932-27b6-4ac5-8420-926b7277f4a3-client-ca-bundle\") pod \"metrics-server-67d9d56b4b-545xf\" (UID: \"840db932-27b6-4ac5-8420-926b7277f4a3\") " pod="openshift-monitoring/metrics-server-67d9d56b4b-545xf"
Sep 17 14:06:28.024410 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:28.024375    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/840db932-27b6-4ac5-8420-926b7277f4a3-audit-log\") pod \"metrics-server-67d9d56b4b-545xf\" (UID: \"840db932-27b6-4ac5-8420-926b7277f4a3\") " pod="openshift-monitoring/metrics-server-67d9d56b4b-545xf"
Sep 17 14:06:28.024771 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:28.024750    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/840db932-27b6-4ac5-8420-926b7277f4a3-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-67d9d56b4b-545xf\" (UID: \"840db932-27b6-4ac5-8420-926b7277f4a3\") " pod="openshift-monitoring/metrics-server-67d9d56b4b-545xf"
Sep 17 14:06:28.025217 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:28.025193    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/840db932-27b6-4ac5-8420-926b7277f4a3-metrics-server-audit-profiles\") pod \"metrics-server-67d9d56b4b-545xf\" (UID: \"840db932-27b6-4ac5-8420-926b7277f4a3\") " pod="openshift-monitoring/metrics-server-67d9d56b4b-545xf"
Sep 17 14:06:28.026098 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:28.026073    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:28.026546 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:28.026520    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/840db932-27b6-4ac5-8420-926b7277f4a3-secret-metrics-server-client-certs\") pod \"metrics-server-67d9d56b4b-545xf\" (UID: \"840db932-27b6-4ac5-8420-926b7277f4a3\") " pod="openshift-monitoring/metrics-server-67d9d56b4b-545xf"
Sep 17 14:06:28.026849 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:28.026830    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/840db932-27b6-4ac5-8420-926b7277f4a3-secret-metrics-server-tls\") pod \"metrics-server-67d9d56b4b-545xf\" (UID: \"840db932-27b6-4ac5-8420-926b7277f4a3\") " pod="openshift-monitoring/metrics-server-67d9d56b4b-545xf"
Sep 17 14:06:28.026898 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:28.026830    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/840db932-27b6-4ac5-8420-926b7277f4a3-client-ca-bundle\") pod \"metrics-server-67d9d56b4b-545xf\" (UID: \"840db932-27b6-4ac5-8420-926b7277f4a3\") " pod="openshift-monitoring/metrics-server-67d9d56b4b-545xf"
Sep 17 14:06:28.033247 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:28.033225    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xbqz\" (UniqueName: \"kubernetes.io/projected/840db932-27b6-4ac5-8420-926b7277f4a3-kube-api-access-5xbqz\") pod \"metrics-server-67d9d56b4b-545xf\" (UID: \"840db932-27b6-4ac5-8420-926b7277f4a3\") " pod="openshift-monitoring/metrics-server-67d9d56b4b-545xf"
Sep 17 14:06:28.102249 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:28.102174    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-67d9d56b4b-545xf"
Sep 17 14:06:28.124775 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:28.124747    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wtl5\" (UniqueName: \"kubernetes.io/projected/214c3411-12f9-49c7-9bcd-01eca91e612b-kube-api-access-6wtl5\") pod \"monitoring-plugin-5d85778b9d-tcmcl\" (UID: \"214c3411-12f9-49c7-9bcd-01eca91e612b\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-tcmcl"
Sep 17 14:06:28.124899 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:28.124864    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/214c3411-12f9-49c7-9bcd-01eca91e612b-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-tcmcl\" (UID: \"214c3411-12f9-49c7-9bcd-01eca91e612b\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-tcmcl"
Sep 17 14:06:28.226240 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:28.226202    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wtl5\" (UniqueName: \"kubernetes.io/projected/214c3411-12f9-49c7-9bcd-01eca91e612b-kube-api-access-6wtl5\") pod \"monitoring-plugin-5d85778b9d-tcmcl\" (UID: \"214c3411-12f9-49c7-9bcd-01eca91e612b\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-tcmcl"
Sep 17 14:06:28.226435 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:28.226292    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/214c3411-12f9-49c7-9bcd-01eca91e612b-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-tcmcl\" (UID: \"214c3411-12f9-49c7-9bcd-01eca91e612b\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-tcmcl"
Sep 17 14:06:28.226492 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:06:28.226435    2582 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Sep 17 14:06:28.226544 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:06:28.226512    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/214c3411-12f9-49c7-9bcd-01eca91e612b-monitoring-plugin-cert podName:214c3411-12f9-49c7-9bcd-01eca91e612b nodeName:}" failed. No retries permitted until 2026-09-17 14:06:28.726489753 +0000 UTC m=+89.728033205 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/214c3411-12f9-49c7-9bcd-01eca91e612b-monitoring-plugin-cert") pod "monitoring-plugin-5d85778b9d-tcmcl" (UID: "214c3411-12f9-49c7-9bcd-01eca91e612b") : secret "monitoring-plugin-cert" not found
Sep 17 14:06:28.236912 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:28.236878    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wtl5\" (UniqueName: \"kubernetes.io/projected/214c3411-12f9-49c7-9bcd-01eca91e612b-kube-api-access-6wtl5\") pod \"monitoring-plugin-5d85778b9d-tcmcl\" (UID: \"214c3411-12f9-49c7-9bcd-01eca91e612b\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-tcmcl"
Sep 17 14:06:28.385910 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:28.385838    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-67d9d56b4b-545xf"]
Sep 17 14:06:28.388914 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:06:28.388888    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod840db932_27b6_4ac5_8420_926b7277f4a3.slice/crio-54052e109db148c8af966bc11e1a2cf9fd28ba1e929ca167bcb622553de68b05 WatchSource:0}: Error finding container 54052e109db148c8af966bc11e1a2cf9fd28ba1e929ca167bcb622553de68b05: Status 404 returned error can't find the container with id 54052e109db148c8af966bc11e1a2cf9fd28ba1e929ca167bcb622553de68b05
Sep 17 14:06:28.623328 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:28.623293    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:28.731512 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:28.731478    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/214c3411-12f9-49c7-9bcd-01eca91e612b-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-tcmcl\" (UID: \"214c3411-12f9-49c7-9bcd-01eca91e612b\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-tcmcl"
Sep 17 14:06:28.733930 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:28.733907    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/214c3411-12f9-49c7-9bcd-01eca91e612b-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-tcmcl\" (UID: \"214c3411-12f9-49c7-9bcd-01eca91e612b\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-tcmcl"
Sep 17 14:06:28.854990 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:28.854956    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-tcmcl"
Sep 17 14:06:28.963086 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:28.963050    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-67d9d56b4b-545xf" event={"ID":"840db932-27b6-4ac5-8420-926b7277f4a3","Type":"ContainerStarted","Data":"54052e109db148c8af966bc11e1a2cf9fd28ba1e929ca167bcb622553de68b05"}
Sep 17 14:06:29.151624 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:29.151536    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-tcmcl"]
Sep 17 14:06:29.154954 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:06:29.154918    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod214c3411_12f9_49c7_9bcd_01eca91e612b.slice/crio-51c960a1d9c7b1124aebb32eae53c5dabf25a8314a6953b7d786a562cc277942 WatchSource:0}: Error finding container 51c960a1d9c7b1124aebb32eae53c5dabf25a8314a6953b7d786a562cc277942: Status 404 returned error can't find the container with id 51c960a1d9c7b1124aebb32eae53c5dabf25a8314a6953b7d786a562cc277942
Sep 17 14:06:29.666804 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:29.666778    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:29.970505 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:29.970458    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-67d9d56b4b-545xf" event={"ID":"840db932-27b6-4ac5-8420-926b7277f4a3","Type":"ContainerStarted","Data":"bd9f1d1c28101c73cb9c668b3b23f838f226c57ccf07b309cd6b065bf9ed77fe"}
Sep 17 14:06:29.972293 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:29.972265    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-tcmcl" event={"ID":"214c3411-12f9-49c7-9bcd-01eca91e612b","Type":"ContainerStarted","Data":"51c960a1d9c7b1124aebb32eae53c5dabf25a8314a6953b7d786a562cc277942"}
Sep 17 14:06:30.021862 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:30.021798    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-67d9d56b4b-545xf" podStartSLOduration=1.749878525 podStartE2EDuration="3.021775088s" totalImagesPullingTime="1.271896563s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:06:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:06:30.021314103 +0000 UTC m=+91.022857584" watchObservedRunningTime="2026-09-17 14:06:30.021775088 +0000 UTC m=+91.023318562"
Sep 17 14:06:30.487578 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:30.487551    2582 scope.go:117] "RemoveContainer" containerID="38cac992b3d877cea1215d81be03306d6b9879d7b116edc03daae89cbdd280de"
Sep 17 14:06:30.978247 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:30.978210    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-tcmcl" event={"ID":"214c3411-12f9-49c7-9bcd-01eca91e612b","Type":"ContainerStarted","Data":"3dc7fe508470621d4c8a317c51b0fb6e2d38e513ccf9c45bb3ec120a271f54eb"}
Sep 17 14:06:30.978709 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:30.978420    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-tcmcl"
Sep 17 14:06:30.980363 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:30.980341    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-rr97k_45ffa13d-635a-46e8-8894-30902b9cf719/console-operator/2.log"
Sep 17 14:06:30.980538 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:30.980458    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-rr97k" event={"ID":"45ffa13d-635a-46e8-8894-30902b9cf719","Type":"ContainerStarted","Data":"994f0de9df925e817ac666f22807cd7077fc94889ab6fbedea8b48dcb57e2154"}
Sep 17 14:06:30.980846 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:30.980812    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-rr97k"
Sep 17 14:06:30.983498 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:30.983478    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-tcmcl"
Sep 17 14:06:30.998742 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:30.998700    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-tcmcl" podStartSLOduration=2.641061233 podStartE2EDuration="3.998686279s" totalImagesPullingTime="1.357625046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:06:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:06:30.99728362 +0000 UTC m=+91.998827094" watchObservedRunningTime="2026-09-17 14:06:30.998686279 +0000 UTC m=+92.000229753"
Sep 17 14:06:31.020648 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:31.020584    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-696884fb9c-rr97k" podStartSLOduration=49.820560957 podStartE2EDuration="56.020567642s" totalImagesPullingTime="6.200006685s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:05:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:06:31.020322905 +0000 UTC m=+92.021866381" watchObservedRunningTime="2026-09-17 14:06:31.020567642 +0000 UTC m=+92.022111116"
Sep 17 14:06:31.111348 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:31.111323    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-696884fb9c-rr97k"
Sep 17 14:06:31.312461 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:31.312359    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-76f99584f4-82kkv"]
Sep 17 14:06:31.315009 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:31.314987    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-82kkv"
Sep 17 14:06:31.317463 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:31.317443    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 17 14:06:31.317582 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:31.317561    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-zljm6\""
Sep 17 14:06:31.317665 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:31.317583    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 17 14:06:31.330240 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:31.330216    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-82kkv"]
Sep 17 14:06:31.351195 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:31.351167    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:31.355389 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:31.355365    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fr67r\" (UniqueName: \"kubernetes.io/projected/c061e677-4341-4e7f-baa8-5d41b143d332-kube-api-access-fr67r\") pod \"downloads-76f99584f4-82kkv\" (UID: \"c061e677-4341-4e7f-baa8-5d41b143d332\") " pod="openshift-console/downloads-76f99584f4-82kkv"
Sep 17 14:06:31.455759 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:31.455729    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fr67r\" (UniqueName: \"kubernetes.io/projected/c061e677-4341-4e7f-baa8-5d41b143d332-kube-api-access-fr67r\") pod \"downloads-76f99584f4-82kkv\" (UID: \"c061e677-4341-4e7f-baa8-5d41b143d332\") " pod="openshift-console/downloads-76f99584f4-82kkv"
Sep 17 14:06:31.466791 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:31.466768    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fr67r\" (UniqueName: \"kubernetes.io/projected/c061e677-4341-4e7f-baa8-5d41b143d332-kube-api-access-fr67r\") pod \"downloads-76f99584f4-82kkv\" (UID: \"c061e677-4341-4e7f-baa8-5d41b143d332\") " pod="openshift-console/downloads-76f99584f4-82kkv"
Sep 17 14:06:31.470800 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:31.470770    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:31.623834 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:31.623734    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-82kkv"
Sep 17 14:06:31.642512 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:31.642483    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:31.834280 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:31.834246    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:31.900245 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:31.900159    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-82kkv"]
Sep 17 14:06:31.950353 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:31.950304    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:31.985703 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:31.985673    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-82kkv" event={"ID":"c061e677-4341-4e7f-baa8-5d41b143d332","Type":"ContainerStarted","Data":"59c94bbc72393454e3d3ee597b49e41f0a432ae2d32a0ac47b6e041393702b8f"}
Sep 17 14:06:32.038263 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:32.038233    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:33.219003 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:33.218969    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:33.332592 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:33.332555    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:33.407199 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:33.407166    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:33.495503 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:33.495427    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:36.114992 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:36.114953    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:36.243979 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:36.243937    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:36.622197 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:36.622163    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:36.787504 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:36.787470    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:36.834279 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:36.834244    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-dg4gz"
Sep 17 14:06:37.338544 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:37.338508    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-c7fc9df5d-jl62r"]
Sep 17 14:06:37.466562 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:37.466526    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:37.567276 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:37.567239    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:37.815298 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:37.815261    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:37.910185 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:37.910150    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:39.331899 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:39.331864    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:39.429772 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:39.429736    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:45.271958 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:45.271882    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:45.405891 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:45.405858    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:45.731299 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:45.731261    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:45.915228 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:45.915186    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:48.061247 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:48.061212    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-82kkv" event={"ID":"c061e677-4341-4e7f-baa8-5d41b143d332","Type":"ContainerStarted","Data":"8722e1aaa027efe77b817243f36e5ba2b1e6da09dffac077d12098156f6e3e23"}
Sep 17 14:06:48.061716 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:48.061419    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-76f99584f4-82kkv"
Sep 17 14:06:48.063302 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:48.063271    2582 patch_prober.go:28] interesting pod/downloads-76f99584f4-82kkv container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.132.0.25:8080/\": dial tcp 10.132.0.25:8080: connect: connection refused" start-of-body=
Sep 17 14:06:48.063424 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:48.063335    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-76f99584f4-82kkv" podUID="c061e677-4341-4e7f-baa8-5d41b143d332" containerName="download-server" probeResult="failure" output="Get \"http://10.132.0.25:8080/\": dial tcp 10.132.0.25:8080: connect: connection refused"
Sep 17 14:06:48.093961 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:48.093909    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-76f99584f4-82kkv" podStartSLOduration=1.094975243 podStartE2EDuration="17.093889589s" totalImagesPullingTime="15.998914346s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:06:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:06:48.092059883 +0000 UTC m=+109.093603357" watchObservedRunningTime="2026-09-17 14:06:48.093889589 +0000 UTC m=+109.095433064"
Sep 17 14:06:48.103159 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:48.103128    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-67d9d56b4b-545xf"
Sep 17 14:06:48.103316 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:48.103206    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-67d9d56b4b-545xf"
Sep 17 14:06:49.081104 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:49.081067    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-76f99584f4-82kkv"
Sep 17 14:06:49.289926 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:49.289889    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:49.448732 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:49.448695    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:50.630732 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:50.630694    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7f8b555fbd-gqz9k"]
Sep 17 14:06:50.642381 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:50.642353    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f8b555fbd-gqz9k"
Sep 17 14:06:50.646108 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:50.646080    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 17 14:06:50.646262 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:50.646113    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-nwvjf\""
Sep 17 14:06:50.646262 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:50.646122    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 17 14:06:50.646262 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:50.646082    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 17 14:06:50.646262 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:50.646175    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 17 14:06:50.646262 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:50.646209    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 17 14:06:50.656229 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:50.656205    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 17 14:06:50.659065 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:50.659038    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f8b555fbd-gqz9k"]
Sep 17 14:06:50.723659 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:50.723595    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e3e546ff-42c7-4aa6-a45d-d9f42d09878d-console-config\") pod \"console-7f8b555fbd-gqz9k\" (UID: \"e3e546ff-42c7-4aa6-a45d-d9f42d09878d\") " pod="openshift-console/console-7f8b555fbd-gqz9k"
Sep 17 14:06:50.723841 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:50.723676    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e3e546ff-42c7-4aa6-a45d-d9f42d09878d-console-oauth-config\") pod \"console-7f8b555fbd-gqz9k\" (UID: \"e3e546ff-42c7-4aa6-a45d-d9f42d09878d\") " pod="openshift-console/console-7f8b555fbd-gqz9k"
Sep 17 14:06:50.723841 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:50.723757    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e3e546ff-42c7-4aa6-a45d-d9f42d09878d-console-serving-cert\") pod \"console-7f8b555fbd-gqz9k\" (UID: \"e3e546ff-42c7-4aa6-a45d-d9f42d09878d\") " pod="openshift-console/console-7f8b555fbd-gqz9k"
Sep 17 14:06:50.723841 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:50.723788    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e3e546ff-42c7-4aa6-a45d-d9f42d09878d-service-ca\") pod \"console-7f8b555fbd-gqz9k\" (UID: \"e3e546ff-42c7-4aa6-a45d-d9f42d09878d\") " pod="openshift-console/console-7f8b555fbd-gqz9k"
Sep 17 14:06:50.723841 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:50.723807    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkhk5\" (UniqueName: \"kubernetes.io/projected/e3e546ff-42c7-4aa6-a45d-d9f42d09878d-kube-api-access-bkhk5\") pod \"console-7f8b555fbd-gqz9k\" (UID: \"e3e546ff-42c7-4aa6-a45d-d9f42d09878d\") " pod="openshift-console/console-7f8b555fbd-gqz9k"
Sep 17 14:06:50.723841 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:50.723829    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e3e546ff-42c7-4aa6-a45d-d9f42d09878d-oauth-serving-cert\") pod \"console-7f8b555fbd-gqz9k\" (UID: \"e3e546ff-42c7-4aa6-a45d-d9f42d09878d\") " pod="openshift-console/console-7f8b555fbd-gqz9k"
Sep 17 14:06:50.724048 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:50.723909    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3e546ff-42c7-4aa6-a45d-d9f42d09878d-trusted-ca-bundle\") pod \"console-7f8b555fbd-gqz9k\" (UID: \"e3e546ff-42c7-4aa6-a45d-d9f42d09878d\") " pod="openshift-console/console-7f8b555fbd-gqz9k"
Sep 17 14:06:50.825045 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:50.825005    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e3e546ff-42c7-4aa6-a45d-d9f42d09878d-console-serving-cert\") pod \"console-7f8b555fbd-gqz9k\" (UID: \"e3e546ff-42c7-4aa6-a45d-d9f42d09878d\") " pod="openshift-console/console-7f8b555fbd-gqz9k"
Sep 17 14:06:50.825226 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:50.825060    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e3e546ff-42c7-4aa6-a45d-d9f42d09878d-service-ca\") pod \"console-7f8b555fbd-gqz9k\" (UID: \"e3e546ff-42c7-4aa6-a45d-d9f42d09878d\") " pod="openshift-console/console-7f8b555fbd-gqz9k"
Sep 17 14:06:50.825226 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:50.825086    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkhk5\" (UniqueName: \"kubernetes.io/projected/e3e546ff-42c7-4aa6-a45d-d9f42d09878d-kube-api-access-bkhk5\") pod \"console-7f8b555fbd-gqz9k\" (UID: \"e3e546ff-42c7-4aa6-a45d-d9f42d09878d\") " pod="openshift-console/console-7f8b555fbd-gqz9k"
Sep 17 14:06:50.825226 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:50.825120    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e3e546ff-42c7-4aa6-a45d-d9f42d09878d-oauth-serving-cert\") pod \"console-7f8b555fbd-gqz9k\" (UID: \"e3e546ff-42c7-4aa6-a45d-d9f42d09878d\") " pod="openshift-console/console-7f8b555fbd-gqz9k"
Sep 17 14:06:50.825226 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:50.825165    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3e546ff-42c7-4aa6-a45d-d9f42d09878d-trusted-ca-bundle\") pod \"console-7f8b555fbd-gqz9k\" (UID: \"e3e546ff-42c7-4aa6-a45d-d9f42d09878d\") " pod="openshift-console/console-7f8b555fbd-gqz9k"
Sep 17 14:06:50.825226 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:50.825204    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e3e546ff-42c7-4aa6-a45d-d9f42d09878d-console-config\") pod \"console-7f8b555fbd-gqz9k\" (UID: \"e3e546ff-42c7-4aa6-a45d-d9f42d09878d\") " pod="openshift-console/console-7f8b555fbd-gqz9k"
Sep 17 14:06:50.825483 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:50.825236    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e3e546ff-42c7-4aa6-a45d-d9f42d09878d-console-oauth-config\") pod \"console-7f8b555fbd-gqz9k\" (UID: \"e3e546ff-42c7-4aa6-a45d-d9f42d09878d\") " pod="openshift-console/console-7f8b555fbd-gqz9k"
Sep 17 14:06:50.826180 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:50.826150    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e3e546ff-42c7-4aa6-a45d-d9f42d09878d-console-config\") pod \"console-7f8b555fbd-gqz9k\" (UID: \"e3e546ff-42c7-4aa6-a45d-d9f42d09878d\") " pod="openshift-console/console-7f8b555fbd-gqz9k"
Sep 17 14:06:50.826315 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:50.826282    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3e546ff-42c7-4aa6-a45d-d9f42d09878d-trusted-ca-bundle\") pod \"console-7f8b555fbd-gqz9k\" (UID: \"e3e546ff-42c7-4aa6-a45d-d9f42d09878d\") " pod="openshift-console/console-7f8b555fbd-gqz9k"
Sep 17 14:06:50.826822 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:50.826797    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e3e546ff-42c7-4aa6-a45d-d9f42d09878d-oauth-serving-cert\") pod \"console-7f8b555fbd-gqz9k\" (UID: \"e3e546ff-42c7-4aa6-a45d-d9f42d09878d\") " pod="openshift-console/console-7f8b555fbd-gqz9k"
Sep 17 14:06:50.826934 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:50.826906    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e3e546ff-42c7-4aa6-a45d-d9f42d09878d-service-ca\") pod \"console-7f8b555fbd-gqz9k\" (UID: \"e3e546ff-42c7-4aa6-a45d-d9f42d09878d\") " pod="openshift-console/console-7f8b555fbd-gqz9k"
Sep 17 14:06:50.828189 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:50.828164    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e3e546ff-42c7-4aa6-a45d-d9f42d09878d-console-oauth-config\") pod \"console-7f8b555fbd-gqz9k\" (UID: \"e3e546ff-42c7-4aa6-a45d-d9f42d09878d\") " pod="openshift-console/console-7f8b555fbd-gqz9k"
Sep 17 14:06:50.828296 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:50.828173    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e3e546ff-42c7-4aa6-a45d-d9f42d09878d-console-serving-cert\") pod \"console-7f8b555fbd-gqz9k\" (UID: \"e3e546ff-42c7-4aa6-a45d-d9f42d09878d\") " pod="openshift-console/console-7f8b555fbd-gqz9k"
Sep 17 14:06:50.836042 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:50.836019    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkhk5\" (UniqueName: \"kubernetes.io/projected/e3e546ff-42c7-4aa6-a45d-d9f42d09878d-kube-api-access-bkhk5\") pod \"console-7f8b555fbd-gqz9k\" (UID: \"e3e546ff-42c7-4aa6-a45d-d9f42d09878d\") " pod="openshift-console/console-7f8b555fbd-gqz9k"
Sep 17 14:06:50.961619 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:50.961567    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f8b555fbd-gqz9k"
Sep 17 14:06:51.278173 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:51.278088    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f8b555fbd-gqz9k"]
Sep 17 14:06:51.281733 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:06:51.281696    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3e546ff_42c7_4aa6_a45d_d9f42d09878d.slice/crio-e365ff7b542ddf756c9749851842727c4da9cb0a21fc8e12de1b794cb090f434 WatchSource:0}: Error finding container e365ff7b542ddf756c9749851842727c4da9cb0a21fc8e12de1b794cb090f434: Status 404 returned error can't find the container with id e365ff7b542ddf756c9749851842727c4da9cb0a21fc8e12de1b794cb090f434
Sep 17 14:06:52.083351 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:52.083270    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f8b555fbd-gqz9k" event={"ID":"e3e546ff-42c7-4aa6-a45d-d9f42d09878d","Type":"ContainerStarted","Data":"e365ff7b542ddf756c9749851842727c4da9cb0a21fc8e12de1b794cb090f434"}
Sep 17 14:06:53.439397 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:53.439301    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:53.605420 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:53.605387    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:55.101108 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:55.101056    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f8b555fbd-gqz9k" event={"ID":"e3e546ff-42c7-4aa6-a45d-d9f42d09878d","Type":"ContainerStarted","Data":"3f77a24f672c34dfc4318cec98c07c8ebaff356250312f1b6be5507e34431a15"}
Sep 17 14:06:55.103002 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:55.102971    2582 generic.go:356] "Generic (PLEG): container finished" podID="9857487e-aa03-45ac-88e4-e6a40b699fef" containerID="f857606855ed2c3128ca7e15faf31cd915185c35137e37d36b1d13d6c4dd18cc" exitCode=0
Sep 17 14:06:55.103151 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:55.102997    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wxbtg" event={"ID":"9857487e-aa03-45ac-88e4-e6a40b699fef","Type":"ContainerDied","Data":"f857606855ed2c3128ca7e15faf31cd915185c35137e37d36b1d13d6c4dd18cc"}
Sep 17 14:06:55.103358 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:55.103343    2582 scope.go:117] "RemoveContainer" containerID="f857606855ed2c3128ca7e15faf31cd915185c35137e37d36b1d13d6c4dd18cc"
Sep 17 14:06:55.156160 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:55.156100    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7f8b555fbd-gqz9k" podStartSLOduration=1.571540986 podStartE2EDuration="5.156082282s" totalImagesPullingTime="3.584541296s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:06:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:06:55.127307679 +0000 UTC m=+116.128851154" watchObservedRunningTime="2026-09-17 14:06:55.156082282 +0000 UTC m=+116.157625755"
Sep 17 14:06:56.109894 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:56.109851    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-wxbtg" event={"ID":"9857487e-aa03-45ac-88e4-e6a40b699fef","Type":"ContainerStarted","Data":"a1c6a4e3b64a7aad5cba831646369a0325707aabf90e42f3b88b9458aa36fff4"}
Sep 17 14:06:56.664512 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:56.664475    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:56.755272 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:56.755235    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:58.538093 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:58.538052    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:58.641934 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:58.641871    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:06:59.125409 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:59.125320    2582 generic.go:356] "Generic (PLEG): container finished" podID="a51d828f-334b-4be5-93f3-33eabaa37533" containerID="9db16563a02168b3f4564c9f6ae295720da4a4e94effaba3ff2a044d1cbae7cf" exitCode=0
Sep 17 14:06:59.125409 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:59.125352    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jl6xc" event={"ID":"a51d828f-334b-4be5-93f3-33eabaa37533","Type":"ContainerDied","Data":"9db16563a02168b3f4564c9f6ae295720da4a4e94effaba3ff2a044d1cbae7cf"}
Sep 17 14:06:59.125732 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:06:59.125717    2582 scope.go:117] "RemoveContainer" containerID="9db16563a02168b3f4564c9f6ae295720da4a4e94effaba3ff2a044d1cbae7cf"
Sep 17 14:07:00.131162 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:00.131128    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jl6xc" event={"ID":"a51d828f-334b-4be5-93f3-33eabaa37533","Type":"ContainerStarted","Data":"7f1b3c90cf43fcf4a7fc70d59623975f6d9e567a4d197dbd781c739209a393f0"}
Sep 17 14:07:00.962125 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:00.962089    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7f8b555fbd-gqz9k"
Sep 17 14:07:00.962311 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:00.962146    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7f8b555fbd-gqz9k"
Sep 17 14:07:00.967017 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:00.966993    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7f8b555fbd-gqz9k"
Sep 17 14:07:01.139261 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:01.139232    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7f8b555fbd-gqz9k"
Sep 17 14:07:01.350644 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:01.350547    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:07:01.450499 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:01.450463    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:07:01.630342 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:01.630256    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:07:01.738089 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:01.738037    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:07:02.360338 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:02.360286    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-c7fc9df5d-jl62r" podUID="81211573-af3a-4b68-ac05-92fdbb7cb8b2" containerName="registry" containerID="cri-o://6e30c1240602b0f15d6c029b67af55eb4dbdc27e212231d43128eec7ccffb34d" gracePeriod=30
Sep 17 14:07:02.631544 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:02.631472    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c7fc9df5d-jl62r"
Sep 17 14:07:02.732672 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:02.732631    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/81211573-af3a-4b68-ac05-92fdbb7cb8b2-registry-certificates\") pod \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\" (UID: \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\") "
Sep 17 14:07:02.732869 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:02.732705    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4bs8r\" (UniqueName: \"kubernetes.io/projected/81211573-af3a-4b68-ac05-92fdbb7cb8b2-kube-api-access-4bs8r\") pod \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\" (UID: \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\") "
Sep 17 14:07:02.732869 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:02.732733    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/81211573-af3a-4b68-ac05-92fdbb7cb8b2-installation-pull-secrets\") pod \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\" (UID: \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\") "
Sep 17 14:07:02.732869 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:02.732773    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/81211573-af3a-4b68-ac05-92fdbb7cb8b2-ca-trust-extracted\") pod \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\" (UID: \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\") "
Sep 17 14:07:02.733029 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:02.732893    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/81211573-af3a-4b68-ac05-92fdbb7cb8b2-trusted-ca\") pod \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\" (UID: \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\") "
Sep 17 14:07:02.733029 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:02.732931    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/81211573-af3a-4b68-ac05-92fdbb7cb8b2-registry-tls\") pod \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\" (UID: \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\") "
Sep 17 14:07:02.733029 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:02.732973    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81211573-af3a-4b68-ac05-92fdbb7cb8b2-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "81211573-af3a-4b68-ac05-92fdbb7cb8b2" (UID: "81211573-af3a-4b68-ac05-92fdbb7cb8b2"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:07:02.733185 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:02.733053    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/81211573-af3a-4b68-ac05-92fdbb7cb8b2-bound-sa-token\") pod \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\" (UID: \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\") "
Sep 17 14:07:02.733185 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:02.733101    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/81211573-af3a-4b68-ac05-92fdbb7cb8b2-image-registry-private-configuration\") pod \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\" (UID: \"81211573-af3a-4b68-ac05-92fdbb7cb8b2\") "
Sep 17 14:07:02.733355 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:02.733336    2582 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/81211573-af3a-4b68-ac05-92fdbb7cb8b2-registry-certificates\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:07:02.733422 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:02.733342    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81211573-af3a-4b68-ac05-92fdbb7cb8b2-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "81211573-af3a-4b68-ac05-92fdbb7cb8b2" (UID: "81211573-af3a-4b68-ac05-92fdbb7cb8b2"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:07:02.735516 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:02.735485    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81211573-af3a-4b68-ac05-92fdbb7cb8b2-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "81211573-af3a-4b68-ac05-92fdbb7cb8b2" (UID: "81211573-af3a-4b68-ac05-92fdbb7cb8b2"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:07:02.735678 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:02.735574    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81211573-af3a-4b68-ac05-92fdbb7cb8b2-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "81211573-af3a-4b68-ac05-92fdbb7cb8b2" (UID: "81211573-af3a-4b68-ac05-92fdbb7cb8b2"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:07:02.735678 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:02.735654    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81211573-af3a-4b68-ac05-92fdbb7cb8b2-kube-api-access-4bs8r" (OuterVolumeSpecName: "kube-api-access-4bs8r") pod "81211573-af3a-4b68-ac05-92fdbb7cb8b2" (UID: "81211573-af3a-4b68-ac05-92fdbb7cb8b2"). InnerVolumeSpecName "kube-api-access-4bs8r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:07:02.735776 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:02.735736    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81211573-af3a-4b68-ac05-92fdbb7cb8b2-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "81211573-af3a-4b68-ac05-92fdbb7cb8b2" (UID: "81211573-af3a-4b68-ac05-92fdbb7cb8b2"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:07:02.735870 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:02.735846    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81211573-af3a-4b68-ac05-92fdbb7cb8b2-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "81211573-af3a-4b68-ac05-92fdbb7cb8b2" (UID: "81211573-af3a-4b68-ac05-92fdbb7cb8b2"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:07:02.740910 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:02.740879    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81211573-af3a-4b68-ac05-92fdbb7cb8b2-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "81211573-af3a-4b68-ac05-92fdbb7cb8b2" (UID: "81211573-af3a-4b68-ac05-92fdbb7cb8b2"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:07:02.834562 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:02.834512    2582 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/81211573-af3a-4b68-ac05-92fdbb7cb8b2-ca-trust-extracted\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:07:02.834562 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:02.834561    2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/81211573-af3a-4b68-ac05-92fdbb7cb8b2-trusted-ca\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:07:02.834562 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:02.834571    2582 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/81211573-af3a-4b68-ac05-92fdbb7cb8b2-registry-tls\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:07:02.834810 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:02.834579    2582 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/81211573-af3a-4b68-ac05-92fdbb7cb8b2-bound-sa-token\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:07:02.834810 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:02.834589    2582 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/81211573-af3a-4b68-ac05-92fdbb7cb8b2-image-registry-private-configuration\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:07:02.834810 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:02.834599    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4bs8r\" (UniqueName: \"kubernetes.io/projected/81211573-af3a-4b68-ac05-92fdbb7cb8b2-kube-api-access-4bs8r\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:07:02.834810 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:02.834625    2582 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/81211573-af3a-4b68-ac05-92fdbb7cb8b2-installation-pull-secrets\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:07:03.145498 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:03.145464    2582 generic.go:356] "Generic (PLEG): container finished" podID="81211573-af3a-4b68-ac05-92fdbb7cb8b2" containerID="6e30c1240602b0f15d6c029b67af55eb4dbdc27e212231d43128eec7ccffb34d" exitCode=0
Sep 17 14:07:03.145680 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:03.145524    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c7fc9df5d-jl62r"
Sep 17 14:07:03.145680 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:03.145547    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c7fc9df5d-jl62r" event={"ID":"81211573-af3a-4b68-ac05-92fdbb7cb8b2","Type":"ContainerDied","Data":"6e30c1240602b0f15d6c029b67af55eb4dbdc27e212231d43128eec7ccffb34d"}
Sep 17 14:07:03.145680 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:03.145644    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c7fc9df5d-jl62r" event={"ID":"81211573-af3a-4b68-ac05-92fdbb7cb8b2","Type":"ContainerDied","Data":"df8e453457a8d9a1dd5a74ea035ca5277fcd74ea2a70972d4a3a5e372c402366"}
Sep 17 14:07:03.145680 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:03.145671    2582 scope.go:117] "RemoveContainer" containerID="6e30c1240602b0f15d6c029b67af55eb4dbdc27e212231d43128eec7ccffb34d"
Sep 17 14:07:03.173159 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:03.173131    2582 scope.go:117] "RemoveContainer" containerID="6e30c1240602b0f15d6c029b67af55eb4dbdc27e212231d43128eec7ccffb34d"
Sep 17 14:07:03.173861 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:07:03.173775    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e30c1240602b0f15d6c029b67af55eb4dbdc27e212231d43128eec7ccffb34d\": container with ID starting with 6e30c1240602b0f15d6c029b67af55eb4dbdc27e212231d43128eec7ccffb34d not found: ID does not exist" containerID="6e30c1240602b0f15d6c029b67af55eb4dbdc27e212231d43128eec7ccffb34d"
Sep 17 14:07:03.173861 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:03.173815    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e30c1240602b0f15d6c029b67af55eb4dbdc27e212231d43128eec7ccffb34d"} err="failed to get container status \"6e30c1240602b0f15d6c029b67af55eb4dbdc27e212231d43128eec7ccffb34d\": rpc error: code = NotFound desc = could not find container \"6e30c1240602b0f15d6c029b67af55eb4dbdc27e212231d43128eec7ccffb34d\": container with ID starting with 6e30c1240602b0f15d6c029b67af55eb4dbdc27e212231d43128eec7ccffb34d not found: ID does not exist"
Sep 17 14:07:03.191099 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:03.191067    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-c7fc9df5d-jl62r"]
Sep 17 14:07:03.198100 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:03.198073    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-c7fc9df5d-jl62r"]
Sep 17 14:07:03.492214 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:03.492178    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81211573-af3a-4b68-ac05-92fdbb7cb8b2" path="/var/lib/kubelet/pods/81211573-af3a-4b68-ac05-92fdbb7cb8b2/volumes"
Sep 17 14:07:08.108219 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:08.108190    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-67d9d56b4b-545xf"
Sep 17 14:07:08.112037 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:08.111990    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-67d9d56b4b-545xf"
Sep 17 14:07:08.421117 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:08.421040    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:07:08.512795 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:08.512763    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:07:14.682351 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:14.682315    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:07:14.773220 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:14.773181    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:07:19.215786 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:19.215752    2582 generic.go:356] "Generic (PLEG): container finished" podID="903d323b-ec37-4e01-89e9-7ab8caa061d7" containerID="c61bae455811e4033751a0fe9d4a391241a0a097946f3155f89a93f02ef262ee" exitCode=0
Sep 17 14:07:19.216157 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:19.215823    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-thrb2" event={"ID":"903d323b-ec37-4e01-89e9-7ab8caa061d7","Type":"ContainerDied","Data":"c61bae455811e4033751a0fe9d4a391241a0a097946f3155f89a93f02ef262ee"}
Sep 17 14:07:19.216243 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:19.216178    2582 scope.go:117] "RemoveContainer" containerID="c61bae455811e4033751a0fe9d4a391241a0a097946f3155f89a93f02ef262ee"
Sep 17 14:07:20.201281 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:20.201220    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:07:20.221973 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:20.221939    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-thrb2" event={"ID":"903d323b-ec37-4e01-89e9-7ab8caa061d7","Type":"ContainerStarted","Data":"603e5181e48e36b51cee44220b2a2e313b6e4d9b6efc50ce844bab1fdae2f9b1"}
Sep 17 14:07:20.301272 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:20.301222    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:07:23.401653 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:23.401595    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:07:23.492781 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:23.492745    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:07:29.634111 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:29.634076    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:07:42.648556 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:42.648486    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:07:43.081661 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:43.081593    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:07:47.823070 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:47.823032    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7cc4cdf7fc-ngzrk"]
Sep 17 14:07:47.823796 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:47.823772    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81211573-af3a-4b68-ac05-92fdbb7cb8b2" containerName="registry"
Sep 17 14:07:47.823796 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:47.823797    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="81211573-af3a-4b68-ac05-92fdbb7cb8b2" containerName="registry"
Sep 17 14:07:47.823956 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:47.823881    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="81211573-af3a-4b68-ac05-92fdbb7cb8b2" containerName="registry"
Sep 17 14:07:47.827579 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:47.827555    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7cc4cdf7fc-ngzrk"
Sep 17 14:07:47.830445 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:47.830425    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 17 14:07:47.830826 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:47.830807    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-wrw8l\""
Sep 17 14:07:47.830886 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:47.830841    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 17 14:07:47.831145 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:47.831131    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 17 14:07:47.831442 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:47.831427    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 17 14:07:47.831729 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:47.831709    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 17 14:07:47.839332 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:47.839313    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 17 14:07:47.845935 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:47.845910    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7cc4cdf7fc-ngzrk"]
Sep 17 14:07:47.906663 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:47.906625    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/1c33a78b-d753-426f-bbd0-10acad2ee10b-secret-telemeter-client\") pod \"telemeter-client-7cc4cdf7fc-ngzrk\" (UID: \"1c33a78b-d753-426f-bbd0-10acad2ee10b\") " pod="openshift-monitoring/telemeter-client-7cc4cdf7fc-ngzrk"
Sep 17 14:07:47.906663 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:47.906669    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c33a78b-d753-426f-bbd0-10acad2ee10b-serving-certs-ca-bundle\") pod \"telemeter-client-7cc4cdf7fc-ngzrk\" (UID: \"1c33a78b-d753-426f-bbd0-10acad2ee10b\") " pod="openshift-monitoring/telemeter-client-7cc4cdf7fc-ngzrk"
Sep 17 14:07:47.906939 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:47.906688    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c33a78b-d753-426f-bbd0-10acad2ee10b-metrics-client-ca\") pod \"telemeter-client-7cc4cdf7fc-ngzrk\" (UID: \"1c33a78b-d753-426f-bbd0-10acad2ee10b\") " pod="openshift-monitoring/telemeter-client-7cc4cdf7fc-ngzrk"
Sep 17 14:07:47.906939 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:47.906769    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/1c33a78b-d753-426f-bbd0-10acad2ee10b-telemeter-client-tls\") pod \"telemeter-client-7cc4cdf7fc-ngzrk\" (UID: \"1c33a78b-d753-426f-bbd0-10acad2ee10b\") " pod="openshift-monitoring/telemeter-client-7cc4cdf7fc-ngzrk"
Sep 17 14:07:47.906939 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:47.906812    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1c33a78b-d753-426f-bbd0-10acad2ee10b-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7cc4cdf7fc-ngzrk\" (UID: \"1c33a78b-d753-426f-bbd0-10acad2ee10b\") " pod="openshift-monitoring/telemeter-client-7cc4cdf7fc-ngzrk"
Sep 17 14:07:47.906939 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:47.906837    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85wxh\" (UniqueName: \"kubernetes.io/projected/1c33a78b-d753-426f-bbd0-10acad2ee10b-kube-api-access-85wxh\") pod \"telemeter-client-7cc4cdf7fc-ngzrk\" (UID: \"1c33a78b-d753-426f-bbd0-10acad2ee10b\") " pod="openshift-monitoring/telemeter-client-7cc4cdf7fc-ngzrk"
Sep 17 14:07:47.906939 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:47.906862    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c33a78b-d753-426f-bbd0-10acad2ee10b-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7cc4cdf7fc-ngzrk\" (UID: \"1c33a78b-d753-426f-bbd0-10acad2ee10b\") " pod="openshift-monitoring/telemeter-client-7cc4cdf7fc-ngzrk"
Sep 17 14:07:47.906939 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:47.906890    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/1c33a78b-d753-426f-bbd0-10acad2ee10b-federate-client-tls\") pod \"telemeter-client-7cc4cdf7fc-ngzrk\" (UID: \"1c33a78b-d753-426f-bbd0-10acad2ee10b\") " pod="openshift-monitoring/telemeter-client-7cc4cdf7fc-ngzrk"
Sep 17 14:07:48.007845 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:48.007807    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/1c33a78b-d753-426f-bbd0-10acad2ee10b-secret-telemeter-client\") pod \"telemeter-client-7cc4cdf7fc-ngzrk\" (UID: \"1c33a78b-d753-426f-bbd0-10acad2ee10b\") " pod="openshift-monitoring/telemeter-client-7cc4cdf7fc-ngzrk"
Sep 17 14:07:48.008056 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:48.007861    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c33a78b-d753-426f-bbd0-10acad2ee10b-serving-certs-ca-bundle\") pod \"telemeter-client-7cc4cdf7fc-ngzrk\" (UID: \"1c33a78b-d753-426f-bbd0-10acad2ee10b\") " pod="openshift-monitoring/telemeter-client-7cc4cdf7fc-ngzrk"
Sep 17 14:07:48.008056 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:48.007890    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c33a78b-d753-426f-bbd0-10acad2ee10b-metrics-client-ca\") pod \"telemeter-client-7cc4cdf7fc-ngzrk\" (UID: \"1c33a78b-d753-426f-bbd0-10acad2ee10b\") " pod="openshift-monitoring/telemeter-client-7cc4cdf7fc-ngzrk"
Sep 17 14:07:48.008056 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:48.007925    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/1c33a78b-d753-426f-bbd0-10acad2ee10b-telemeter-client-tls\") pod \"telemeter-client-7cc4cdf7fc-ngzrk\" (UID: \"1c33a78b-d753-426f-bbd0-10acad2ee10b\") " pod="openshift-monitoring/telemeter-client-7cc4cdf7fc-ngzrk"
Sep 17 14:07:48.008056 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:48.007988    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1c33a78b-d753-426f-bbd0-10acad2ee10b-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7cc4cdf7fc-ngzrk\" (UID: \"1c33a78b-d753-426f-bbd0-10acad2ee10b\") " pod="openshift-monitoring/telemeter-client-7cc4cdf7fc-ngzrk"
Sep 17 14:07:48.008056 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:48.008019    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85wxh\" (UniqueName: \"kubernetes.io/projected/1c33a78b-d753-426f-bbd0-10acad2ee10b-kube-api-access-85wxh\") pod \"telemeter-client-7cc4cdf7fc-ngzrk\" (UID: \"1c33a78b-d753-426f-bbd0-10acad2ee10b\") " pod="openshift-monitoring/telemeter-client-7cc4cdf7fc-ngzrk"
Sep 17 14:07:48.008056 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:48.008045    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c33a78b-d753-426f-bbd0-10acad2ee10b-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7cc4cdf7fc-ngzrk\" (UID: \"1c33a78b-d753-426f-bbd0-10acad2ee10b\") " pod="openshift-monitoring/telemeter-client-7cc4cdf7fc-ngzrk"
Sep 17 14:07:48.008366 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:48.008074    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/1c33a78b-d753-426f-bbd0-10acad2ee10b-federate-client-tls\") pod \"telemeter-client-7cc4cdf7fc-ngzrk\" (UID: \"1c33a78b-d753-426f-bbd0-10acad2ee10b\") " pod="openshift-monitoring/telemeter-client-7cc4cdf7fc-ngzrk"
Sep 17 14:07:48.008803 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:48.008775    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c33a78b-d753-426f-bbd0-10acad2ee10b-metrics-client-ca\") pod \"telemeter-client-7cc4cdf7fc-ngzrk\" (UID: \"1c33a78b-d753-426f-bbd0-10acad2ee10b\") " pod="openshift-monitoring/telemeter-client-7cc4cdf7fc-ngzrk"
Sep 17 14:07:48.008992 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:48.008873    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c33a78b-d753-426f-bbd0-10acad2ee10b-serving-certs-ca-bundle\") pod \"telemeter-client-7cc4cdf7fc-ngzrk\" (UID: \"1c33a78b-d753-426f-bbd0-10acad2ee10b\") " pod="openshift-monitoring/telemeter-client-7cc4cdf7fc-ngzrk"
Sep 17 14:07:48.009097 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:48.008937    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c33a78b-d753-426f-bbd0-10acad2ee10b-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7cc4cdf7fc-ngzrk\" (UID: \"1c33a78b-d753-426f-bbd0-10acad2ee10b\") " pod="openshift-monitoring/telemeter-client-7cc4cdf7fc-ngzrk"
Sep 17 14:07:48.010670 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:48.010646    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/1c33a78b-d753-426f-bbd0-10acad2ee10b-secret-telemeter-client\") pod \"telemeter-client-7cc4cdf7fc-ngzrk\" (UID: \"1c33a78b-d753-426f-bbd0-10acad2ee10b\") " pod="openshift-monitoring/telemeter-client-7cc4cdf7fc-ngzrk"
Sep 17 14:07:48.010757 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:48.010652    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/1c33a78b-d753-426f-bbd0-10acad2ee10b-federate-client-tls\") pod \"telemeter-client-7cc4cdf7fc-ngzrk\" (UID: \"1c33a78b-d753-426f-bbd0-10acad2ee10b\") " pod="openshift-monitoring/telemeter-client-7cc4cdf7fc-ngzrk"
Sep 17 14:07:48.010961 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:48.010937    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1c33a78b-d753-426f-bbd0-10acad2ee10b-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7cc4cdf7fc-ngzrk\" (UID: \"1c33a78b-d753-426f-bbd0-10acad2ee10b\") " pod="openshift-monitoring/telemeter-client-7cc4cdf7fc-ngzrk"
Sep 17 14:07:48.011000 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:48.010984    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/1c33a78b-d753-426f-bbd0-10acad2ee10b-telemeter-client-tls\") pod \"telemeter-client-7cc4cdf7fc-ngzrk\" (UID: \"1c33a78b-d753-426f-bbd0-10acad2ee10b\") " pod="openshift-monitoring/telemeter-client-7cc4cdf7fc-ngzrk"
Sep 17 14:07:48.017470 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:48.017448    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85wxh\" (UniqueName: \"kubernetes.io/projected/1c33a78b-d753-426f-bbd0-10acad2ee10b-kube-api-access-85wxh\") pod \"telemeter-client-7cc4cdf7fc-ngzrk\" (UID: \"1c33a78b-d753-426f-bbd0-10acad2ee10b\") " pod="openshift-monitoring/telemeter-client-7cc4cdf7fc-ngzrk"
Sep 17 14:07:48.137298 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:48.137209    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7cc4cdf7fc-ngzrk"
Sep 17 14:07:48.441776 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:48.441699    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7cc4cdf7fc-ngzrk"]
Sep 17 14:07:48.445291 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:07:48.445260    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c33a78b_d753_426f_bbd0_10acad2ee10b.slice/crio-4ff9676283f49f6a2491fcc4f6e7c77a6d33ede7d5f872e7ea0dbb8e6c0e2fc4 WatchSource:0}: Error finding container 4ff9676283f49f6a2491fcc4f6e7c77a6d33ede7d5f872e7ea0dbb8e6c0e2fc4: Status 404 returned error can't find the container with id 4ff9676283f49f6a2491fcc4f6e7c77a6d33ede7d5f872e7ea0dbb8e6c0e2fc4
Sep 17 14:07:49.361071 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:49.360955    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7cc4cdf7fc-ngzrk" event={"ID":"1c33a78b-d753-426f-bbd0-10acad2ee10b","Type":"ContainerStarted","Data":"4ff9676283f49f6a2491fcc4f6e7c77a6d33ede7d5f872e7ea0dbb8e6c0e2fc4"}
Sep 17 14:07:50.369999 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:50.369909    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7cc4cdf7fc-ngzrk" event={"ID":"1c33a78b-d753-426f-bbd0-10acad2ee10b","Type":"ContainerStarted","Data":"37b5498bdaea73f27d118881dc427497c5683dc64e8e9dda8438ccf1f95eb0e7"}
Sep 17 14:07:52.381042 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:52.380956    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7cc4cdf7fc-ngzrk" event={"ID":"1c33a78b-d753-426f-bbd0-10acad2ee10b","Type":"ContainerStarted","Data":"6831913f1718818f052ff14770cf02c738753b59a6492edb9c2b774a276f032e"}
Sep 17 14:07:52.381042 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:52.381013    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7cc4cdf7fc-ngzrk" event={"ID":"1c33a78b-d753-426f-bbd0-10acad2ee10b","Type":"ContainerStarted","Data":"35c7b0cc4f8271976a29e05176aae715f04861c8c0869f3d7b7079bb297128f8"}
Sep 17 14:07:52.420030 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:52.419968    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7cc4cdf7fc-ngzrk" podStartSLOduration=2.502711253 podStartE2EDuration="5.419946385s" totalImagesPullingTime="2.917235132s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:07:47 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:07:52.418240743 +0000 UTC m=+173.419784220" watchObservedRunningTime="2026-09-17 14:07:52.419946385 +0000 UTC m=+173.421489862"
Sep 17 14:07:52.811026 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:52.810992    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:07:53.205164 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:53.205116    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:07:53.418639 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:53.418586    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:07:53.628935 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:07:53.628824    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:08:03.646362 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:08:03.646314    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f8b555fbd-gqz9k"]
Sep 17 14:08:03.804213 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:08:03.804180    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:08:03.904940 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:08:03.904857    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:08:04.005525 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:08:04.005490    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:08:28.666373 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:08:28.666314    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7f8b555fbd-gqz9k" podUID="e3e546ff-42c7-4aa6-a45d-d9f42d09878d" containerName="console" containerID="cri-o://3f77a24f672c34dfc4318cec98c07c8ebaff356250312f1b6be5507e34431a15" gracePeriod=15
Sep 17 14:08:28.917153 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:08:28.917093    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f8b555fbd-gqz9k"
Sep 17 14:08:28.955745 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:08:28.955706    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e3e546ff-42c7-4aa6-a45d-d9f42d09878d-console-oauth-config\") pod \"e3e546ff-42c7-4aa6-a45d-d9f42d09878d\" (UID: \"e3e546ff-42c7-4aa6-a45d-d9f42d09878d\") "
Sep 17 14:08:28.955891 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:08:28.955760    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bkhk5\" (UniqueName: \"kubernetes.io/projected/e3e546ff-42c7-4aa6-a45d-d9f42d09878d-kube-api-access-bkhk5\") pod \"e3e546ff-42c7-4aa6-a45d-d9f42d09878d\" (UID: \"e3e546ff-42c7-4aa6-a45d-d9f42d09878d\") "
Sep 17 14:08:28.955891 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:08:28.955799    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e3e546ff-42c7-4aa6-a45d-d9f42d09878d-oauth-serving-cert\") pod \"e3e546ff-42c7-4aa6-a45d-d9f42d09878d\" (UID: \"e3e546ff-42c7-4aa6-a45d-d9f42d09878d\") "
Sep 17 14:08:28.955891 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:08:28.955877    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3e546ff-42c7-4aa6-a45d-d9f42d09878d-trusted-ca-bundle\") pod \"e3e546ff-42c7-4aa6-a45d-d9f42d09878d\" (UID: \"e3e546ff-42c7-4aa6-a45d-d9f42d09878d\") "
Sep 17 14:08:28.956099 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:08:28.955906    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e3e546ff-42c7-4aa6-a45d-d9f42d09878d-console-serving-cert\") pod \"e3e546ff-42c7-4aa6-a45d-d9f42d09878d\" (UID: \"e3e546ff-42c7-4aa6-a45d-d9f42d09878d\") "
Sep 17 14:08:28.956099 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:08:28.955931    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e3e546ff-42c7-4aa6-a45d-d9f42d09878d-service-ca\") pod \"e3e546ff-42c7-4aa6-a45d-d9f42d09878d\" (UID: \"e3e546ff-42c7-4aa6-a45d-d9f42d09878d\") "
Sep 17 14:08:28.956367 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:08:28.956255    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e3e546ff-42c7-4aa6-a45d-d9f42d09878d-console-config\") pod \"e3e546ff-42c7-4aa6-a45d-d9f42d09878d\" (UID: \"e3e546ff-42c7-4aa6-a45d-d9f42d09878d\") "
Sep 17 14:08:28.956367 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:08:28.956315    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3e546ff-42c7-4aa6-a45d-d9f42d09878d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e3e546ff-42c7-4aa6-a45d-d9f42d09878d" (UID: "e3e546ff-42c7-4aa6-a45d-d9f42d09878d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:08:28.956367 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:08:28.956323    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3e546ff-42c7-4aa6-a45d-d9f42d09878d-service-ca" (OuterVolumeSpecName: "service-ca") pod "e3e546ff-42c7-4aa6-a45d-d9f42d09878d" (UID: "e3e546ff-42c7-4aa6-a45d-d9f42d09878d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:08:28.956367 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:08:28.956330    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3e546ff-42c7-4aa6-a45d-d9f42d09878d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e3e546ff-42c7-4aa6-a45d-d9f42d09878d" (UID: "e3e546ff-42c7-4aa6-a45d-d9f42d09878d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:08:28.956772 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:08:28.956513    2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3e546ff-42c7-4aa6-a45d-d9f42d09878d-trusted-ca-bundle\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:08:28.956772 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:08:28.956532    2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e3e546ff-42c7-4aa6-a45d-d9f42d09878d-service-ca\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:08:28.956772 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:08:28.956546    2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e3e546ff-42c7-4aa6-a45d-d9f42d09878d-oauth-serving-cert\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:08:28.956772 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:08:28.956747    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3e546ff-42c7-4aa6-a45d-d9f42d09878d-console-config" (OuterVolumeSpecName: "console-config") pod "e3e546ff-42c7-4aa6-a45d-d9f42d09878d" (UID: "e3e546ff-42c7-4aa6-a45d-d9f42d09878d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:08:28.958239 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:08:28.958206    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3e546ff-42c7-4aa6-a45d-d9f42d09878d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e3e546ff-42c7-4aa6-a45d-d9f42d09878d" (UID: "e3e546ff-42c7-4aa6-a45d-d9f42d09878d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:08:28.958411 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:08:28.958392    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3e546ff-42c7-4aa6-a45d-d9f42d09878d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e3e546ff-42c7-4aa6-a45d-d9f42d09878d" (UID: "e3e546ff-42c7-4aa6-a45d-d9f42d09878d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:08:28.958477 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:08:28.958394    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3e546ff-42c7-4aa6-a45d-d9f42d09878d-kube-api-access-bkhk5" (OuterVolumeSpecName: "kube-api-access-bkhk5") pod "e3e546ff-42c7-4aa6-a45d-d9f42d09878d" (UID: "e3e546ff-42c7-4aa6-a45d-d9f42d09878d"). InnerVolumeSpecName "kube-api-access-bkhk5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:08:29.057682 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:08:29.057652    2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e3e546ff-42c7-4aa6-a45d-d9f42d09878d-console-oauth-config\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:08:29.057682 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:08:29.057677    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bkhk5\" (UniqueName: \"kubernetes.io/projected/e3e546ff-42c7-4aa6-a45d-d9f42d09878d-kube-api-access-bkhk5\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:08:29.057682 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:08:29.057688    2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e3e546ff-42c7-4aa6-a45d-d9f42d09878d-console-serving-cert\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:08:29.057910 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:08:29.057697    2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e3e546ff-42c7-4aa6-a45d-d9f42d09878d-console-config\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:08:29.541309 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:08:29.541270    2582 generic.go:356] "Generic (PLEG): container finished" podID="e3e546ff-42c7-4aa6-a45d-d9f42d09878d" containerID="3f77a24f672c34dfc4318cec98c07c8ebaff356250312f1b6be5507e34431a15" exitCode=0
Sep 17 14:08:29.541469 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:08:29.541321    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f8b555fbd-gqz9k"
Sep 17 14:08:29.541469 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:08:29.541353    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f8b555fbd-gqz9k" event={"ID":"e3e546ff-42c7-4aa6-a45d-d9f42d09878d","Type":"ContainerDied","Data":"3f77a24f672c34dfc4318cec98c07c8ebaff356250312f1b6be5507e34431a15"}
Sep 17 14:08:29.541469 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:08:29.541405    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f8b555fbd-gqz9k" event={"ID":"e3e546ff-42c7-4aa6-a45d-d9f42d09878d","Type":"ContainerDied","Data":"e365ff7b542ddf756c9749851842727c4da9cb0a21fc8e12de1b794cb090f434"}
Sep 17 14:08:29.541469 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:08:29.541426    2582 scope.go:117] "RemoveContainer" containerID="3f77a24f672c34dfc4318cec98c07c8ebaff356250312f1b6be5507e34431a15"
Sep 17 14:08:29.549694 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:08:29.549676    2582 scope.go:117] "RemoveContainer" containerID="3f77a24f672c34dfc4318cec98c07c8ebaff356250312f1b6be5507e34431a15"
Sep 17 14:08:29.549997 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:08:29.549978    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f77a24f672c34dfc4318cec98c07c8ebaff356250312f1b6be5507e34431a15\": container with ID starting with 3f77a24f672c34dfc4318cec98c07c8ebaff356250312f1b6be5507e34431a15 not found: ID does not exist" containerID="3f77a24f672c34dfc4318cec98c07c8ebaff356250312f1b6be5507e34431a15"
Sep 17 14:08:29.550040 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:08:29.550004    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f77a24f672c34dfc4318cec98c07c8ebaff356250312f1b6be5507e34431a15"} err="failed to get container status \"3f77a24f672c34dfc4318cec98c07c8ebaff356250312f1b6be5507e34431a15\": rpc error: code = NotFound desc = could not find container \"3f77a24f672c34dfc4318cec98c07c8ebaff356250312f1b6be5507e34431a15\": container with ID starting with 3f77a24f672c34dfc4318cec98c07c8ebaff356250312f1b6be5507e34431a15 not found: ID does not exist"
Sep 17 14:08:29.558706 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:08:29.558681    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f8b555fbd-gqz9k"]
Sep 17 14:08:29.564840 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:08:29.564818    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7f8b555fbd-gqz9k"]
Sep 17 14:08:31.492260 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:08:31.492229    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3e546ff-42c7-4aa6-a45d-d9f42d09878d" path="/var/lib/kubelet/pods/e3e546ff-42c7-4aa6-a45d-d9f42d09878d/volumes"
Sep 17 14:08:50.799190 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:08:50.799155    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:08:50.917223 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:08:50.917191    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:08:57.995186 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:08:57.995155    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:08:58.168200 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:08:58.168150    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:09:08.799641 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:08.799575    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-64585d4699-8csw4"]
Sep 17 14:09:08.800237 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:08.800167    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3e546ff-42c7-4aa6-a45d-d9f42d09878d" containerName="console"
Sep 17 14:09:08.800237 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:08.800192    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3e546ff-42c7-4aa6-a45d-d9f42d09878d" containerName="console"
Sep 17 14:09:08.800361 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:08.800285    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3e546ff-42c7-4aa6-a45d-d9f42d09878d" containerName="console"
Sep 17 14:09:08.804602 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:08.804579    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64585d4699-8csw4"
Sep 17 14:09:08.807113 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:08.807092    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 17 14:09:08.807245 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:08.807092    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 17 14:09:08.808036 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:08.807999    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 17 14:09:08.808240 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:08.808057    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 17 14:09:08.808240 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:08.808060    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-nwvjf\""
Sep 17 14:09:08.808240 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:08.808062    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 17 14:09:08.812749 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:08.812525    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 17 14:09:08.816825 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:08.816804    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64585d4699-8csw4"]
Sep 17 14:09:08.871523 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:08.871488    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:09:08.972093 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:08.972058    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:09:08.979249 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:08.979227    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b34d9da8-57dc-43b0-9d2a-3894a00a6df9-console-serving-cert\") pod \"console-64585d4699-8csw4\" (UID: \"b34d9da8-57dc-43b0-9d2a-3894a00a6df9\") " pod="openshift-console/console-64585d4699-8csw4"
Sep 17 14:09:08.979303 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:08.979253    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b34d9da8-57dc-43b0-9d2a-3894a00a6df9-service-ca\") pod \"console-64585d4699-8csw4\" (UID: \"b34d9da8-57dc-43b0-9d2a-3894a00a6df9\") " pod="openshift-console/console-64585d4699-8csw4"
Sep 17 14:09:08.979303 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:08.979270    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b34d9da8-57dc-43b0-9d2a-3894a00a6df9-console-config\") pod \"console-64585d4699-8csw4\" (UID: \"b34d9da8-57dc-43b0-9d2a-3894a00a6df9\") " pod="openshift-console/console-64585d4699-8csw4"
Sep 17 14:09:08.979303 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:08.979292    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smzj2\" (UniqueName: \"kubernetes.io/projected/b34d9da8-57dc-43b0-9d2a-3894a00a6df9-kube-api-access-smzj2\") pod \"console-64585d4699-8csw4\" (UID: \"b34d9da8-57dc-43b0-9d2a-3894a00a6df9\") " pod="openshift-console/console-64585d4699-8csw4"
Sep 17 14:09:08.979421 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:08.979403    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b34d9da8-57dc-43b0-9d2a-3894a00a6df9-console-oauth-config\") pod \"console-64585d4699-8csw4\" (UID: \"b34d9da8-57dc-43b0-9d2a-3894a00a6df9\") " pod="openshift-console/console-64585d4699-8csw4"
Sep 17 14:09:08.979457 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:08.979424    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b34d9da8-57dc-43b0-9d2a-3894a00a6df9-trusted-ca-bundle\") pod \"console-64585d4699-8csw4\" (UID: \"b34d9da8-57dc-43b0-9d2a-3894a00a6df9\") " pod="openshift-console/console-64585d4699-8csw4"
Sep 17 14:09:08.979457 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:08.979443    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b34d9da8-57dc-43b0-9d2a-3894a00a6df9-oauth-serving-cert\") pod \"console-64585d4699-8csw4\" (UID: \"b34d9da8-57dc-43b0-9d2a-3894a00a6df9\") " pod="openshift-console/console-64585d4699-8csw4"
Sep 17 14:09:09.080329 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:09.080256    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b34d9da8-57dc-43b0-9d2a-3894a00a6df9-console-config\") pod \"console-64585d4699-8csw4\" (UID: \"b34d9da8-57dc-43b0-9d2a-3894a00a6df9\") " pod="openshift-console/console-64585d4699-8csw4"
Sep 17 14:09:09.080329 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:09.080289    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smzj2\" (UniqueName: \"kubernetes.io/projected/b34d9da8-57dc-43b0-9d2a-3894a00a6df9-kube-api-access-smzj2\") pod \"console-64585d4699-8csw4\" (UID: \"b34d9da8-57dc-43b0-9d2a-3894a00a6df9\") " pod="openshift-console/console-64585d4699-8csw4"
Sep 17 14:09:09.080544 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:09.080337    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b34d9da8-57dc-43b0-9d2a-3894a00a6df9-console-oauth-config\") pod \"console-64585d4699-8csw4\" (UID: \"b34d9da8-57dc-43b0-9d2a-3894a00a6df9\") " pod="openshift-console/console-64585d4699-8csw4"
Sep 17 14:09:09.080544 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:09.080354    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b34d9da8-57dc-43b0-9d2a-3894a00a6df9-trusted-ca-bundle\") pod \"console-64585d4699-8csw4\" (UID: \"b34d9da8-57dc-43b0-9d2a-3894a00a6df9\") " pod="openshift-console/console-64585d4699-8csw4"
Sep 17 14:09:09.080544 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:09.080371    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b34d9da8-57dc-43b0-9d2a-3894a00a6df9-oauth-serving-cert\") pod \"console-64585d4699-8csw4\" (UID: \"b34d9da8-57dc-43b0-9d2a-3894a00a6df9\") " pod="openshift-console/console-64585d4699-8csw4"
Sep 17 14:09:09.080544 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:09.080407    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b34d9da8-57dc-43b0-9d2a-3894a00a6df9-console-serving-cert\") pod \"console-64585d4699-8csw4\" (UID: \"b34d9da8-57dc-43b0-9d2a-3894a00a6df9\") " pod="openshift-console/console-64585d4699-8csw4"
Sep 17 14:09:09.080544 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:09.080427    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b34d9da8-57dc-43b0-9d2a-3894a00a6df9-service-ca\") pod \"console-64585d4699-8csw4\" (UID: \"b34d9da8-57dc-43b0-9d2a-3894a00a6df9\") " pod="openshift-console/console-64585d4699-8csw4"
Sep 17 14:09:09.081153 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:09.081125    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b34d9da8-57dc-43b0-9d2a-3894a00a6df9-console-config\") pod \"console-64585d4699-8csw4\" (UID: \"b34d9da8-57dc-43b0-9d2a-3894a00a6df9\") " pod="openshift-console/console-64585d4699-8csw4"
Sep 17 14:09:09.081283 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:09.081125    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b34d9da8-57dc-43b0-9d2a-3894a00a6df9-oauth-serving-cert\") pod \"console-64585d4699-8csw4\" (UID: \"b34d9da8-57dc-43b0-9d2a-3894a00a6df9\") " pod="openshift-console/console-64585d4699-8csw4"
Sep 17 14:09:09.081283 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:09.081164    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b34d9da8-57dc-43b0-9d2a-3894a00a6df9-service-ca\") pod \"console-64585d4699-8csw4\" (UID: \"b34d9da8-57dc-43b0-9d2a-3894a00a6df9\") " pod="openshift-console/console-64585d4699-8csw4"
Sep 17 14:09:09.081404 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:09.081294    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b34d9da8-57dc-43b0-9d2a-3894a00a6df9-trusted-ca-bundle\") pod \"console-64585d4699-8csw4\" (UID: \"b34d9da8-57dc-43b0-9d2a-3894a00a6df9\") " pod="openshift-console/console-64585d4699-8csw4"
Sep 17 14:09:09.083055 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:09.083023    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b34d9da8-57dc-43b0-9d2a-3894a00a6df9-console-serving-cert\") pod \"console-64585d4699-8csw4\" (UID: \"b34d9da8-57dc-43b0-9d2a-3894a00a6df9\") " pod="openshift-console/console-64585d4699-8csw4"
Sep 17 14:09:09.083055 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:09.083040    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b34d9da8-57dc-43b0-9d2a-3894a00a6df9-console-oauth-config\") pod \"console-64585d4699-8csw4\" (UID: \"b34d9da8-57dc-43b0-9d2a-3894a00a6df9\") " pod="openshift-console/console-64585d4699-8csw4"
Sep 17 14:09:09.088176 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:09.088155    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smzj2\" (UniqueName: \"kubernetes.io/projected/b34d9da8-57dc-43b0-9d2a-3894a00a6df9-kube-api-access-smzj2\") pod \"console-64585d4699-8csw4\" (UID: \"b34d9da8-57dc-43b0-9d2a-3894a00a6df9\") " pod="openshift-console/console-64585d4699-8csw4"
Sep 17 14:09:09.115178 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:09.115152    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64585d4699-8csw4"
Sep 17 14:09:09.391220 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:09.391192    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64585d4699-8csw4"]
Sep 17 14:09:09.393959 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:09:09.393927    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb34d9da8_57dc_43b0_9d2a_3894a00a6df9.slice/crio-43ff7651e67861a03b52f7fbe8880edd515cd310b0c4a2f0b1edb08536256460 WatchSource:0}: Error finding container 43ff7651e67861a03b52f7fbe8880edd515cd310b0c4a2f0b1edb08536256460: Status 404 returned error can't find the container with id 43ff7651e67861a03b52f7fbe8880edd515cd310b0c4a2f0b1edb08536256460
Sep 17 14:09:09.714533 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:09.714498    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64585d4699-8csw4" event={"ID":"b34d9da8-57dc-43b0-9d2a-3894a00a6df9","Type":"ContainerStarted","Data":"b4cb96a7ef8d4772b92383548e0e9ab49896f936b955969df03caba5592eff9e"}
Sep 17 14:09:09.714722 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:09.714547    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64585d4699-8csw4" event={"ID":"b34d9da8-57dc-43b0-9d2a-3894a00a6df9","Type":"ContainerStarted","Data":"43ff7651e67861a03b52f7fbe8880edd515cd310b0c4a2f0b1edb08536256460"}
Sep 17 14:09:09.735671 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:09.735603    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-64585d4699-8csw4" podStartSLOduration=1.735586809 podStartE2EDuration="1.735586809s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:09:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:09:09.733930472 +0000 UTC m=+250.735473968" watchObservedRunningTime="2026-09-17 14:09:09.735586809 +0000 UTC m=+250.737130356"
Sep 17 14:09:10.824535 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:10.824496    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:09:10.899359 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:10.899325    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:09:12.414769 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:12.414729    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:09:19.116256 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:19.116208    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-64585d4699-8csw4"
Sep 17 14:09:19.116742 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:19.116273    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-64585d4699-8csw4"
Sep 17 14:09:19.121025 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:19.121005    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-64585d4699-8csw4"
Sep 17 14:09:19.760552 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:19.760521    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-64585d4699-8csw4"
Sep 17 14:09:19.789333 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:19.789299    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:09:19.911754 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:19.911721    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:09:20.103599 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:20.103481    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:09:59.367965 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:59.367933    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-rr97k_45ffa13d-635a-46e8-8894-30902b9cf719/console-operator/2.log"
Sep 17 14:09:59.372869 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:59.372836    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-rr97k_45ffa13d-635a-46e8-8894-30902b9cf719/console-operator/2.log"
Sep 17 14:09:59.378852 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:09:59.378829    2582 kubelet.go:1648] "Image garbage collection succeeded"
Sep 17 14:10:10.932145 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:10:10.932104    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:10:13.280033 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:10:13.279998    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:10:15.119913 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:10:15.119858    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:11:12.023477 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:12.023390    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:11:12.346438 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:12.346348    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:11:15.835785 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:15.835730    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:11:15.936784 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:15.936747    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:11:17.644714 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:17.644683    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:11:17.793887 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:17.793849    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:11:17.886690 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:17.886658    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-zj52b"]
Sep 17 14:11:17.893873 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:17.893846    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-zj52b"
Sep 17 14:11:17.896924 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:17.896843    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-lgmhj\""
Sep 17 14:11:17.897133 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:17.897119    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 17 14:11:17.902028 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:17.902004    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 17 14:11:17.914136 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:17.914108    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-zj52b"]
Sep 17 14:11:17.951663 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:17.951621    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:11:17.952834 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:17.952811    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1973e0d9-5932-4bdd-b582-1d058fff37c4-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-zj52b\" (UID: \"1973e0d9-5932-4bdd-b582-1d058fff37c4\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zj52b"
Sep 17 14:11:17.952952 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:17.952935    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdtvs\" (UniqueName: \"kubernetes.io/projected/1973e0d9-5932-4bdd-b582-1d058fff37c4-kube-api-access-wdtvs\") pod \"cert-manager-webhook-6486ffd997-zj52b\" (UID: \"1973e0d9-5932-4bdd-b582-1d058fff37c4\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zj52b"
Sep 17 14:11:18.053716 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:18.053676    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdtvs\" (UniqueName: \"kubernetes.io/projected/1973e0d9-5932-4bdd-b582-1d058fff37c4-kube-api-access-wdtvs\") pod \"cert-manager-webhook-6486ffd997-zj52b\" (UID: \"1973e0d9-5932-4bdd-b582-1d058fff37c4\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zj52b"
Sep 17 14:11:18.053886 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:18.053732    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1973e0d9-5932-4bdd-b582-1d058fff37c4-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-zj52b\" (UID: \"1973e0d9-5932-4bdd-b582-1d058fff37c4\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zj52b"
Sep 17 14:11:18.073782 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:18.073758    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1973e0d9-5932-4bdd-b582-1d058fff37c4-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-zj52b\" (UID: \"1973e0d9-5932-4bdd-b582-1d058fff37c4\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zj52b"
Sep 17 14:11:18.073896 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:18.073824    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdtvs\" (UniqueName: \"kubernetes.io/projected/1973e0d9-5932-4bdd-b582-1d058fff37c4-kube-api-access-wdtvs\") pod \"cert-manager-webhook-6486ffd997-zj52b\" (UID: \"1973e0d9-5932-4bdd-b582-1d058fff37c4\") " pod="cert-manager/cert-manager-webhook-6486ffd997-zj52b"
Sep 17 14:11:18.213909 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:18.213876    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-zj52b"
Sep 17 14:11:18.489944 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:18.489917    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-zj52b"]
Sep 17 14:11:18.492837 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:11:18.492810    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1973e0d9_5932_4bdd_b582_1d058fff37c4.slice/crio-f77a5d447c2a0365cd035f327930c3fcc4b9d0877e7f0fe9a1204a8e51413fa8 WatchSource:0}: Error finding container f77a5d447c2a0365cd035f327930c3fcc4b9d0877e7f0fe9a1204a8e51413fa8: Status 404 returned error can't find the container with id f77a5d447c2a0365cd035f327930c3fcc4b9d0877e7f0fe9a1204a8e51413fa8
Sep 17 14:11:18.495155 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:18.495140    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 14:11:19.272512 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:19.272481    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-zj52b" event={"ID":"1973e0d9-5932-4bdd-b582-1d058fff37c4","Type":"ContainerStarted","Data":"f77a5d447c2a0365cd035f327930c3fcc4b9d0877e7f0fe9a1204a8e51413fa8"}
Sep 17 14:11:20.038493 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:20.038452    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:11:20.465873 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:20.465799    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:11:22.634604 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:22.634521    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:11:22.734436 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:22.734403    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:11:23.296490 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:23.296452    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-zj52b" event={"ID":"1973e0d9-5932-4bdd-b582-1d058fff37c4","Type":"ContainerStarted","Data":"dbaa1aa273a57d38c9effba9756a5923fa2e2843fe5f304381129d32121cafb8"}
Sep 17 14:11:23.296685 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:23.296498    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-zj52b"
Sep 17 14:11:23.334943 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:23.334892    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-zj52b" podStartSLOduration=2.520383215 podStartE2EDuration="6.334878163s" totalImagesPullingTime="3.814494948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:11:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:11:23.332212863 +0000 UTC m=+384.333756337" watchObservedRunningTime="2026-09-17 14:11:23.334878163 +0000 UTC m=+384.336421637"
Sep 17 14:11:27.254645 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:27.254597    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:11:27.669733 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:27.669649    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:11:27.818640 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:27.818589    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:11:28.492589 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:28.492551    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-htkmb"]
Sep 17 14:11:28.495835 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:28.495817    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-htkmb"
Sep 17 14:11:28.498317 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:28.498296    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-cbg9b\""
Sep 17 14:11:28.507256 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:28.507232    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-htkmb"]
Sep 17 14:11:28.547300 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:28.547260    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bf64l\" (UniqueName: \"kubernetes.io/projected/9d24bfca-dfa4-4d41-827e-a00ff90bce9c-kube-api-access-bf64l\") pod \"cert-manager-55b4d954d5-htkmb\" (UID: \"9d24bfca-dfa4-4d41-827e-a00ff90bce9c\") " pod="cert-manager/cert-manager-55b4d954d5-htkmb"
Sep 17 14:11:28.547470 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:28.547322    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9d24bfca-dfa4-4d41-827e-a00ff90bce9c-bound-sa-token\") pod \"cert-manager-55b4d954d5-htkmb\" (UID: \"9d24bfca-dfa4-4d41-827e-a00ff90bce9c\") " pod="cert-manager/cert-manager-55b4d954d5-htkmb"
Sep 17 14:11:28.648230 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:28.648197    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9d24bfca-dfa4-4d41-827e-a00ff90bce9c-bound-sa-token\") pod \"cert-manager-55b4d954d5-htkmb\" (UID: \"9d24bfca-dfa4-4d41-827e-a00ff90bce9c\") " pod="cert-manager/cert-manager-55b4d954d5-htkmb"
Sep 17 14:11:28.648372 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:28.648304    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bf64l\" (UniqueName: \"kubernetes.io/projected/9d24bfca-dfa4-4d41-827e-a00ff90bce9c-kube-api-access-bf64l\") pod \"cert-manager-55b4d954d5-htkmb\" (UID: \"9d24bfca-dfa4-4d41-827e-a00ff90bce9c\") " pod="cert-manager/cert-manager-55b4d954d5-htkmb"
Sep 17 14:11:28.657315 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:28.657284    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9d24bfca-dfa4-4d41-827e-a00ff90bce9c-bound-sa-token\") pod \"cert-manager-55b4d954d5-htkmb\" (UID: \"9d24bfca-dfa4-4d41-827e-a00ff90bce9c\") " pod="cert-manager/cert-manager-55b4d954d5-htkmb"
Sep 17 14:11:28.657428 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:28.657345    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bf64l\" (UniqueName: \"kubernetes.io/projected/9d24bfca-dfa4-4d41-827e-a00ff90bce9c-kube-api-access-bf64l\") pod \"cert-manager-55b4d954d5-htkmb\" (UID: \"9d24bfca-dfa4-4d41-827e-a00ff90bce9c\") " pod="cert-manager/cert-manager-55b4d954d5-htkmb"
Sep 17 14:11:28.804793 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:28.804704    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-htkmb"
Sep 17 14:11:29.075924 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:29.075895    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-htkmb"]
Sep 17 14:11:29.078761 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:11:29.078734    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9d24bfca_dfa4_4d41_827e_a00ff90bce9c.slice/crio-96c300e0505dcd24fda0920411670e2fcec858dd1360cc363ca2e17118c4d071 WatchSource:0}: Error finding container 96c300e0505dcd24fda0920411670e2fcec858dd1360cc363ca2e17118c4d071: Status 404 returned error can't find the container with id 96c300e0505dcd24fda0920411670e2fcec858dd1360cc363ca2e17118c4d071
Sep 17 14:11:29.303376 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:29.303344    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-zj52b"
Sep 17 14:11:29.325460 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:29.325421    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-htkmb" event={"ID":"9d24bfca-dfa4-4d41-827e-a00ff90bce9c","Type":"ContainerStarted","Data":"3b728e233c7e98f56bb414123b2afa16b8662d61ead5a8b96d4d6dc7e9b99dbc"}
Sep 17 14:11:29.325780 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:29.325747    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-htkmb" event={"ID":"9d24bfca-dfa4-4d41-827e-a00ff90bce9c","Type":"ContainerStarted","Data":"96c300e0505dcd24fda0920411670e2fcec858dd1360cc363ca2e17118c4d071"}
Sep 17 14:11:29.346062 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:29.345961    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-htkmb" podStartSLOduration=1.345944201 podStartE2EDuration="1.345944201s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:11:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:11:29.343934238 +0000 UTC m=+390.345477724" watchObservedRunningTime="2026-09-17 14:11:29.345944201 +0000 UTC m=+390.347487674"
Sep 17 14:11:29.509676 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:29.509638    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:11:29.628346 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:29.628247    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:11:36.235835 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:36.235796    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:11:37.254982 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:37.254946    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:11:37.422332 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:37.422296    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:11:42.091360 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:42.091317    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:11:42.193364 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:42.193333    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:11:44.249296 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:44.249258    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:11:54.555964 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:54.555927    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:11:54.608894 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:54.608858    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5cb7bb5755-dvn4b"]
Sep 17 14:11:54.612213 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:54.612191    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5cb7bb5755-dvn4b"
Sep 17 14:11:54.614795 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:54.614773    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 17 14:11:54.614903 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:54.614846    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 17 14:11:54.615819 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:54.615802    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 17 14:11:54.616064 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:54.616050    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 17 14:11:54.616145 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:54.616130    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 17 14:11:54.616206 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:54.616177    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-58dlt\""
Sep 17 14:11:54.622683 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:54.622665    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5cb7bb5755-dvn4b"]
Sep 17 14:11:54.654337 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:54.654305    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:11:54.771174 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:54.771137    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/efa8a65a-f3ef-4179-9891-e8b71eb14708-cert\") pod \"lws-controller-manager-5cb7bb5755-dvn4b\" (UID: \"efa8a65a-f3ef-4179-9891-e8b71eb14708\") " pod="openshift-lws-operator/lws-controller-manager-5cb7bb5755-dvn4b"
Sep 17 14:11:54.771174 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:54.771179    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w497w\" (UniqueName: \"kubernetes.io/projected/efa8a65a-f3ef-4179-9891-e8b71eb14708-kube-api-access-w497w\") pod \"lws-controller-manager-5cb7bb5755-dvn4b\" (UID: \"efa8a65a-f3ef-4179-9891-e8b71eb14708\") " pod="openshift-lws-operator/lws-controller-manager-5cb7bb5755-dvn4b"
Sep 17 14:11:54.771417 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:54.771217    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/efa8a65a-f3ef-4179-9891-e8b71eb14708-manager-config\") pod \"lws-controller-manager-5cb7bb5755-dvn4b\" (UID: \"efa8a65a-f3ef-4179-9891-e8b71eb14708\") " pod="openshift-lws-operator/lws-controller-manager-5cb7bb5755-dvn4b"
Sep 17 14:11:54.771417 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:54.771279    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/efa8a65a-f3ef-4179-9891-e8b71eb14708-metrics-cert\") pod \"lws-controller-manager-5cb7bb5755-dvn4b\" (UID: \"efa8a65a-f3ef-4179-9891-e8b71eb14708\") " pod="openshift-lws-operator/lws-controller-manager-5cb7bb5755-dvn4b"
Sep 17 14:11:54.872042 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:54.871959    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/efa8a65a-f3ef-4179-9891-e8b71eb14708-manager-config\") pod \"lws-controller-manager-5cb7bb5755-dvn4b\" (UID: \"efa8a65a-f3ef-4179-9891-e8b71eb14708\") " pod="openshift-lws-operator/lws-controller-manager-5cb7bb5755-dvn4b"
Sep 17 14:11:54.872042 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:54.871998    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/efa8a65a-f3ef-4179-9891-e8b71eb14708-metrics-cert\") pod \"lws-controller-manager-5cb7bb5755-dvn4b\" (UID: \"efa8a65a-f3ef-4179-9891-e8b71eb14708\") " pod="openshift-lws-operator/lws-controller-manager-5cb7bb5755-dvn4b"
Sep 17 14:11:54.872042 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:54.872036    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/efa8a65a-f3ef-4179-9891-e8b71eb14708-cert\") pod \"lws-controller-manager-5cb7bb5755-dvn4b\" (UID: \"efa8a65a-f3ef-4179-9891-e8b71eb14708\") " pod="openshift-lws-operator/lws-controller-manager-5cb7bb5755-dvn4b"
Sep 17 14:11:54.872301 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:54.872061    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w497w\" (UniqueName: \"kubernetes.io/projected/efa8a65a-f3ef-4179-9891-e8b71eb14708-kube-api-access-w497w\") pod \"lws-controller-manager-5cb7bb5755-dvn4b\" (UID: \"efa8a65a-f3ef-4179-9891-e8b71eb14708\") " pod="openshift-lws-operator/lws-controller-manager-5cb7bb5755-dvn4b"
Sep 17 14:11:54.872791 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:54.872772    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/efa8a65a-f3ef-4179-9891-e8b71eb14708-manager-config\") pod \"lws-controller-manager-5cb7bb5755-dvn4b\" (UID: \"efa8a65a-f3ef-4179-9891-e8b71eb14708\") " pod="openshift-lws-operator/lws-controller-manager-5cb7bb5755-dvn4b"
Sep 17 14:11:54.874528 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:54.874506    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/efa8a65a-f3ef-4179-9891-e8b71eb14708-cert\") pod \"lws-controller-manager-5cb7bb5755-dvn4b\" (UID: \"efa8a65a-f3ef-4179-9891-e8b71eb14708\") " pod="openshift-lws-operator/lws-controller-manager-5cb7bb5755-dvn4b"
Sep 17 14:11:54.874770 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:54.874749    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/efa8a65a-f3ef-4179-9891-e8b71eb14708-metrics-cert\") pod \"lws-controller-manager-5cb7bb5755-dvn4b\" (UID: \"efa8a65a-f3ef-4179-9891-e8b71eb14708\") " pod="openshift-lws-operator/lws-controller-manager-5cb7bb5755-dvn4b"
Sep 17 14:11:54.881916 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:54.881883    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w497w\" (UniqueName: \"kubernetes.io/projected/efa8a65a-f3ef-4179-9891-e8b71eb14708-kube-api-access-w497w\") pod \"lws-controller-manager-5cb7bb5755-dvn4b\" (UID: \"efa8a65a-f3ef-4179-9891-e8b71eb14708\") " pod="openshift-lws-operator/lws-controller-manager-5cb7bb5755-dvn4b"
Sep 17 14:11:54.922242 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:54.922209    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5cb7bb5755-dvn4b"
Sep 17 14:11:55.197988 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:55.197930    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5cb7bb5755-dvn4b"]
Sep 17 14:11:55.200445 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:11:55.200422    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefa8a65a_f3ef_4179_9891_e8b71eb14708.slice/crio-edd162b77556e47b495d1c65ad6cd1286068d44e91446df34d9ab9442fa2087b WatchSource:0}: Error finding container edd162b77556e47b495d1c65ad6cd1286068d44e91446df34d9ab9442fa2087b: Status 404 returned error can't find the container with id edd162b77556e47b495d1c65ad6cd1286068d44e91446df34d9ab9442fa2087b
Sep 17 14:11:55.443202 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:55.443167    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5cb7bb5755-dvn4b" event={"ID":"efa8a65a-f3ef-4179-9891-e8b71eb14708","Type":"ContainerStarted","Data":"edd162b77556e47b495d1c65ad6cd1286068d44e91446df34d9ab9442fa2087b"}
Sep 17 14:11:58.459491 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:58.459452    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5cb7bb5755-dvn4b" event={"ID":"efa8a65a-f3ef-4179-9891-e8b71eb14708","Type":"ContainerStarted","Data":"0cacd033188bbb3bac9fafa8553f2a56d6063fd9967763e02a64aae915a881f2"}
Sep 17 14:11:58.459965 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:58.459576    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5cb7bb5755-dvn4b"
Sep 17 14:11:58.481106 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:11:58.481059    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5cb7bb5755-dvn4b" podStartSLOduration=2.228594381 podStartE2EDuration="4.481045283s" totalImagesPullingTime="2.252450902s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:11:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:11:58.47915469 +0000 UTC m=+419.480698165" watchObservedRunningTime="2026-09-17 14:11:58.481045283 +0000 UTC m=+419.482588758"
Sep 17 14:12:01.505931 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:01.505898    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:12:01.638264 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:01.638224    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:12:02.056054 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:02.056012    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:12:02.264629 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:02.264576    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:12:02.367301 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:02.367223    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:12:02.422209 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:02.422165    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7895599765-fb8q4"]
Sep 17 14:12:02.425728 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:02.425711    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7895599765-fb8q4"
Sep 17 14:12:02.428277 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:02.428249    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-89xvx\""
Sep 17 14:12:02.428277 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:02.428265    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 17 14:12:02.428439 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:02.428284    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 17 14:12:02.428730 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:02.428714    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 17 14:12:02.429426 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:02.429413    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 17 14:12:02.439489 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:02.439467    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7895599765-fb8q4"]
Sep 17 14:12:02.536386 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:02.536352    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/caf36299-408c-4b4c-a5f3-f6125324fc7e-tmp\") pod \"opendatahub-operator-controller-manager-7895599765-fb8q4\" (UID: \"caf36299-408c-4b4c-a5f3-f6125324fc7e\") " pod="opendatahub/opendatahub-operator-controller-manager-7895599765-fb8q4"
Sep 17 14:12:02.536386 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:02.536390    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/caf36299-408c-4b4c-a5f3-f6125324fc7e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7895599765-fb8q4\" (UID: \"caf36299-408c-4b4c-a5f3-f6125324fc7e\") " pod="opendatahub/opendatahub-operator-controller-manager-7895599765-fb8q4"
Sep 17 14:12:02.536808 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:02.536504    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/caf36299-408c-4b4c-a5f3-f6125324fc7e-manifests\") pod \"opendatahub-operator-controller-manager-7895599765-fb8q4\" (UID: \"caf36299-408c-4b4c-a5f3-f6125324fc7e\") " pod="opendatahub/opendatahub-operator-controller-manager-7895599765-fb8q4"
Sep 17 14:12:02.536808 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:02.536534    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmhkk\" (UniqueName: \"kubernetes.io/projected/caf36299-408c-4b4c-a5f3-f6125324fc7e-kube-api-access-fmhkk\") pod \"opendatahub-operator-controller-manager-7895599765-fb8q4\" (UID: \"caf36299-408c-4b4c-a5f3-f6125324fc7e\") " pod="opendatahub/opendatahub-operator-controller-manager-7895599765-fb8q4"
Sep 17 14:12:02.536808 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:02.536567    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/caf36299-408c-4b4c-a5f3-f6125324fc7e-webhook-cert\") pod \"opendatahub-operator-controller-manager-7895599765-fb8q4\" (UID: \"caf36299-408c-4b4c-a5f3-f6125324fc7e\") " pod="opendatahub/opendatahub-operator-controller-manager-7895599765-fb8q4"
Sep 17 14:12:02.637772 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:02.637680    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmhkk\" (UniqueName: \"kubernetes.io/projected/caf36299-408c-4b4c-a5f3-f6125324fc7e-kube-api-access-fmhkk\") pod \"opendatahub-operator-controller-manager-7895599765-fb8q4\" (UID: \"caf36299-408c-4b4c-a5f3-f6125324fc7e\") " pod="opendatahub/opendatahub-operator-controller-manager-7895599765-fb8q4"
Sep 17 14:12:02.637772 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:02.637728    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/caf36299-408c-4b4c-a5f3-f6125324fc7e-webhook-cert\") pod \"opendatahub-operator-controller-manager-7895599765-fb8q4\" (UID: \"caf36299-408c-4b4c-a5f3-f6125324fc7e\") " pod="opendatahub/opendatahub-operator-controller-manager-7895599765-fb8q4"
Sep 17 14:12:02.637772 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:02.637767    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/caf36299-408c-4b4c-a5f3-f6125324fc7e-tmp\") pod \"opendatahub-operator-controller-manager-7895599765-fb8q4\" (UID: \"caf36299-408c-4b4c-a5f3-f6125324fc7e\") " pod="opendatahub/opendatahub-operator-controller-manager-7895599765-fb8q4"
Sep 17 14:12:02.638041 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:02.637800    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/caf36299-408c-4b4c-a5f3-f6125324fc7e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7895599765-fb8q4\" (UID: \"caf36299-408c-4b4c-a5f3-f6125324fc7e\") " pod="opendatahub/opendatahub-operator-controller-manager-7895599765-fb8q4"
Sep 17 14:12:02.638041 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:02.637880    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/caf36299-408c-4b4c-a5f3-f6125324fc7e-manifests\") pod \"opendatahub-operator-controller-manager-7895599765-fb8q4\" (UID: \"caf36299-408c-4b4c-a5f3-f6125324fc7e\") " pod="opendatahub/opendatahub-operator-controller-manager-7895599765-fb8q4"
Sep 17 14:12:02.638281 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:02.638251    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/caf36299-408c-4b4c-a5f3-f6125324fc7e-manifests\") pod \"opendatahub-operator-controller-manager-7895599765-fb8q4\" (UID: \"caf36299-408c-4b4c-a5f3-f6125324fc7e\") " pod="opendatahub/opendatahub-operator-controller-manager-7895599765-fb8q4"
Sep 17 14:12:02.640117 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:02.640090    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/caf36299-408c-4b4c-a5f3-f6125324fc7e-tmp\") pod \"opendatahub-operator-controller-manager-7895599765-fb8q4\" (UID: \"caf36299-408c-4b4c-a5f3-f6125324fc7e\") " pod="opendatahub/opendatahub-operator-controller-manager-7895599765-fb8q4"
Sep 17 14:12:02.640306 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:02.640286    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/caf36299-408c-4b4c-a5f3-f6125324fc7e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7895599765-fb8q4\" (UID: \"caf36299-408c-4b4c-a5f3-f6125324fc7e\") " pod="opendatahub/opendatahub-operator-controller-manager-7895599765-fb8q4"
Sep 17 14:12:02.640348 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:02.640298    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/caf36299-408c-4b4c-a5f3-f6125324fc7e-webhook-cert\") pod \"opendatahub-operator-controller-manager-7895599765-fb8q4\" (UID: \"caf36299-408c-4b4c-a5f3-f6125324fc7e\") " pod="opendatahub/opendatahub-operator-controller-manager-7895599765-fb8q4"
Sep 17 14:12:02.656718 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:02.656693    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmhkk\" (UniqueName: \"kubernetes.io/projected/caf36299-408c-4b4c-a5f3-f6125324fc7e-kube-api-access-fmhkk\") pod \"opendatahub-operator-controller-manager-7895599765-fb8q4\" (UID: \"caf36299-408c-4b4c-a5f3-f6125324fc7e\") " pod="opendatahub/opendatahub-operator-controller-manager-7895599765-fb8q4"
Sep 17 14:12:02.725166 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:02.725130    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:12:02.737163 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:02.737134    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7895599765-fb8q4"
Sep 17 14:12:02.876255 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:02.876220    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:12:02.978452 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:02.978419    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:12:03.019338 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:03.019313    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7895599765-fb8q4"]
Sep 17 14:12:03.022067 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:12:03.022039    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaf36299_408c_4b4c_a5f3_f6125324fc7e.slice/crio-4848e6bcd7f91e2096456320e3f657d5589dada313613eb7f8278caae3320c79 WatchSource:0}: Error finding container 4848e6bcd7f91e2096456320e3f657d5589dada313613eb7f8278caae3320c79: Status 404 returned error can't find the container with id 4848e6bcd7f91e2096456320e3f657d5589dada313613eb7f8278caae3320c79
Sep 17 14:12:03.081339 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:03.081305    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:12:03.481764 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:03.481731    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7895599765-fb8q4" event={"ID":"caf36299-408c-4b4c-a5f3-f6125324fc7e","Type":"ContainerStarted","Data":"4848e6bcd7f91e2096456320e3f657d5589dada313613eb7f8278caae3320c79"}
Sep 17 14:12:04.461828 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:04.461788    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:12:04.549694 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:04.549658    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:12:06.507001 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:06.506965    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7895599765-fb8q4" event={"ID":"caf36299-408c-4b4c-a5f3-f6125324fc7e","Type":"ContainerStarted","Data":"f23a424a850bfc9f5f0b1f732b227c636997ab4411f1970ce03462d64c25ff7d"}
Sep 17 14:12:07.513332 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:07.513299    2582 generic.go:356] "Generic (PLEG): container finished" podID="caf36299-408c-4b4c-a5f3-f6125324fc7e" containerID="f23a424a850bfc9f5f0b1f732b227c636997ab4411f1970ce03462d64c25ff7d" exitCode=0
Sep 17 14:12:07.513737 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:07.513387    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7895599765-fb8q4" event={"ID":"caf36299-408c-4b4c-a5f3-f6125324fc7e","Type":"ContainerDied","Data":"f23a424a850bfc9f5f0b1f732b227c636997ab4411f1970ce03462d64c25ff7d"}
Sep 17 14:12:08.519453 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:08.519424    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7895599765-fb8q4" event={"ID":"caf36299-408c-4b4c-a5f3-f6125324fc7e","Type":"ContainerStarted","Data":"96547e354d2dcc0651e111f3d5c3f0f22644581b5904db26f66f51fec9656621"}
Sep 17 14:12:08.519823 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:08.519528    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7895599765-fb8q4"
Sep 17 14:12:08.561734 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:08.561683    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7895599765-fb8q4" podStartSLOduration=1.656747698 podStartE2EDuration="6.561669337s" totalImagesPullingTime="3.810144659s" totalInitContainerRuntime="1.09477698s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:12:02 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:12:08.560539759 +0000 UTC m=+429.562083233" watchObservedRunningTime="2026-09-17 14:12:08.561669337 +0000 UTC m=+429.563212808"
Sep 17 14:12:09.465628 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:09.465574    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5cb7bb5755-dvn4b"
Sep 17 14:12:13.624807 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:13.624769    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:12:13.725030 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:13.724998    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:12:17.373811 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:17.373778    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:12:17.481587 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:17.481548    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:12:17.790424 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:17.790391    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:12:17.886132 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:17.886100    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:12:19.526418 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:19.526389    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7895599765-fb8q4"
Sep 17 14:12:21.364432 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:21.364398    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:12:23.161821 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:23.161737    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:12:23.298807 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:23.298771    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:12:23.468661 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:23.468631    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:12:23.571473 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:23.571433    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:12:24.659896 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:24.659866    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:12:29.719731 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:29.719698    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:12:29.976829 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:29.976754    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:12:30.365511 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:30.365434    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn"]
Sep 17 14:12:30.368979 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:30.368961    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn"
Sep 17 14:12:30.371349 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:30.371332    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-6mxml\""
Sep 17 14:12:30.371432 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:30.371392    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 17 14:12:30.389909 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:30.389882    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn"]
Sep 17 14:12:30.420371 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:30.420341    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:12:30.466708 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:30.466678    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnfjt\" (UniqueName: \"kubernetes.io/projected/d76458e2-4b38-4f8b-947a-839acc027b9e-kube-api-access-wnfjt\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn\" (UID: \"d76458e2-4b38-4f8b-947a-839acc027b9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn"
Sep 17 14:12:30.466831 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:30.466713    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d76458e2-4b38-4f8b-947a-839acc027b9e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn\" (UID: \"d76458e2-4b38-4f8b-947a-839acc027b9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn"
Sep 17 14:12:30.466831 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:30.466745    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d76458e2-4b38-4f8b-947a-839acc027b9e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn\" (UID: \"d76458e2-4b38-4f8b-947a-839acc027b9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn"
Sep 17 14:12:30.466901 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:30.466830    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d76458e2-4b38-4f8b-947a-839acc027b9e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn\" (UID: \"d76458e2-4b38-4f8b-947a-839acc027b9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn"
Sep 17 14:12:30.466901 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:30.466865    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d76458e2-4b38-4f8b-947a-839acc027b9e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn\" (UID: \"d76458e2-4b38-4f8b-947a-839acc027b9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn"
Sep 17 14:12:30.466901 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:30.466891    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d76458e2-4b38-4f8b-947a-839acc027b9e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn\" (UID: \"d76458e2-4b38-4f8b-947a-839acc027b9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn"
Sep 17 14:12:30.466995 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:30.466938    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d76458e2-4b38-4f8b-947a-839acc027b9e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn\" (UID: \"d76458e2-4b38-4f8b-947a-839acc027b9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn"
Sep 17 14:12:30.467032 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:30.467007    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d76458e2-4b38-4f8b-947a-839acc027b9e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn\" (UID: \"d76458e2-4b38-4f8b-947a-839acc027b9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn"
Sep 17 14:12:30.467065 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:30.467029    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d76458e2-4b38-4f8b-947a-839acc027b9e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn\" (UID: \"d76458e2-4b38-4f8b-947a-839acc027b9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn"
Sep 17 14:12:30.567846 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:30.567821    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d76458e2-4b38-4f8b-947a-839acc027b9e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn\" (UID: \"d76458e2-4b38-4f8b-947a-839acc027b9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn"
Sep 17 14:12:30.567996 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:30.567857    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d76458e2-4b38-4f8b-947a-839acc027b9e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn\" (UID: \"d76458e2-4b38-4f8b-947a-839acc027b9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn"
Sep 17 14:12:30.567996 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:30.567877    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d76458e2-4b38-4f8b-947a-839acc027b9e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn\" (UID: \"d76458e2-4b38-4f8b-947a-839acc027b9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn"
Sep 17 14:12:30.568108 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:30.568001    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d76458e2-4b38-4f8b-947a-839acc027b9e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn\" (UID: \"d76458e2-4b38-4f8b-947a-839acc027b9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn"
Sep 17 14:12:30.568108 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:30.568078    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d76458e2-4b38-4f8b-947a-839acc027b9e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn\" (UID: \"d76458e2-4b38-4f8b-947a-839acc027b9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn"
Sep 17 14:12:30.568219 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:30.568108    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d76458e2-4b38-4f8b-947a-839acc027b9e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn\" (UID: \"d76458e2-4b38-4f8b-947a-839acc027b9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn"
Sep 17 14:12:30.568219 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:30.568180    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnfjt\" (UniqueName: \"kubernetes.io/projected/d76458e2-4b38-4f8b-947a-839acc027b9e-kube-api-access-wnfjt\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn\" (UID: \"d76458e2-4b38-4f8b-947a-839acc027b9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn"
Sep 17 14:12:30.568325 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:30.568240    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d76458e2-4b38-4f8b-947a-839acc027b9e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn\" (UID: \"d76458e2-4b38-4f8b-947a-839acc027b9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn"
Sep 17 14:12:30.568325 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:30.568216    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d76458e2-4b38-4f8b-947a-839acc027b9e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn\" (UID: \"d76458e2-4b38-4f8b-947a-839acc027b9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn"
Sep 17 14:12:30.568325 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:30.568306    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d76458e2-4b38-4f8b-947a-839acc027b9e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn\" (UID: \"d76458e2-4b38-4f8b-947a-839acc027b9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn"
Sep 17 14:12:30.568475 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:30.568406    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d76458e2-4b38-4f8b-947a-839acc027b9e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn\" (UID: \"d76458e2-4b38-4f8b-947a-839acc027b9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn"
Sep 17 14:12:30.568541 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:30.568405    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d76458e2-4b38-4f8b-947a-839acc027b9e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn\" (UID: \"d76458e2-4b38-4f8b-947a-839acc027b9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn"
Sep 17 14:12:30.568681 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:30.568653    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d76458e2-4b38-4f8b-947a-839acc027b9e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn\" (UID: \"d76458e2-4b38-4f8b-947a-839acc027b9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn"
Sep 17 14:12:30.568904 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:30.568885    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d76458e2-4b38-4f8b-947a-839acc027b9e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn\" (UID: \"d76458e2-4b38-4f8b-947a-839acc027b9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn"
Sep 17 14:12:30.570246 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:30.570231    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d76458e2-4b38-4f8b-947a-839acc027b9e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn\" (UID: \"d76458e2-4b38-4f8b-947a-839acc027b9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn"
Sep 17 14:12:30.570492 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:30.570472    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d76458e2-4b38-4f8b-947a-839acc027b9e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn\" (UID: \"d76458e2-4b38-4f8b-947a-839acc027b9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn"
Sep 17 14:12:30.576345 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:30.576319    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d76458e2-4b38-4f8b-947a-839acc027b9e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn\" (UID: \"d76458e2-4b38-4f8b-947a-839acc027b9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn"
Sep 17 14:12:30.576424 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:30.576381    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnfjt\" (UniqueName: \"kubernetes.io/projected/d76458e2-4b38-4f8b-947a-839acc027b9e-kube-api-access-wnfjt\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn\" (UID: \"d76458e2-4b38-4f8b-947a-839acc027b9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn"
Sep 17 14:12:30.642083 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:30.642009    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:12:30.679259 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:30.679232    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn"
Sep 17 14:12:30.744502 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:30.744466    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:12:30.962116 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:30.962090    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn"]
Sep 17 14:12:30.963886 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:12:30.963860    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd76458e2_4b38_4f8b_947a_839acc027b9e.slice/crio-4ea51d41a357bfb59ef27f1357a25789aaa42f65add2a9a514aab53a4635418c WatchSource:0}: Error finding container 4ea51d41a357bfb59ef27f1357a25789aaa42f65add2a9a514aab53a4635418c: Status 404 returned error can't find the container with id 4ea51d41a357bfb59ef27f1357a25789aaa42f65add2a9a514aab53a4635418c
Sep 17 14:12:31.627233 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:31.627198    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn" event={"ID":"d76458e2-4b38-4f8b-947a-839acc027b9e","Type":"ContainerStarted","Data":"4ea51d41a357bfb59ef27f1357a25789aaa42f65add2a9a514aab53a4635418c"}
Sep 17 14:12:32.500677 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:32.500641    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-mdqnr"]
Sep 17 14:12:32.504945 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:32.504926    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-mdqnr"
Sep 17 14:12:32.508177 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:32.508158    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 17 14:12:32.508343 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:32.508322    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-lqk9n\""
Sep 17 14:12:32.524589 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:32.524560    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-mdqnr"]
Sep 17 14:12:32.547050 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:32.547023    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:12:32.588006 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:32.587976    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d61faf19-d78e-494d-928c-27a2ffbe33c0-cert\") pod \"odh-model-controller-7998c67dc4-mdqnr\" (UID: \"d61faf19-d78e-494d-928c-27a2ffbe33c0\") " pod="opendatahub/odh-model-controller-7998c67dc4-mdqnr"
Sep 17 14:12:32.588152 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:32.588098    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5p8v\" (UniqueName: \"kubernetes.io/projected/d61faf19-d78e-494d-928c-27a2ffbe33c0-kube-api-access-n5p8v\") pod \"odh-model-controller-7998c67dc4-mdqnr\" (UID: \"d61faf19-d78e-494d-928c-27a2ffbe33c0\") " pod="opendatahub/odh-model-controller-7998c67dc4-mdqnr"
Sep 17 14:12:32.646256 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:32.646224    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:12:32.689064 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:32.689035    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5p8v\" (UniqueName: \"kubernetes.io/projected/d61faf19-d78e-494d-928c-27a2ffbe33c0-kube-api-access-n5p8v\") pod \"odh-model-controller-7998c67dc4-mdqnr\" (UID: \"d61faf19-d78e-494d-928c-27a2ffbe33c0\") " pod="opendatahub/odh-model-controller-7998c67dc4-mdqnr"
Sep 17 14:12:32.689227 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:32.689095    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d61faf19-d78e-494d-928c-27a2ffbe33c0-cert\") pod \"odh-model-controller-7998c67dc4-mdqnr\" (UID: \"d61faf19-d78e-494d-928c-27a2ffbe33c0\") " pod="opendatahub/odh-model-controller-7998c67dc4-mdqnr"
Sep 17 14:12:32.689227 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:12:32.689181    2582 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 17 14:12:32.689308 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:12:32.689231    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d61faf19-d78e-494d-928c-27a2ffbe33c0-cert podName:d61faf19-d78e-494d-928c-27a2ffbe33c0 nodeName:}" failed. No retries permitted until 2026-09-17 14:12:33.189216227 +0000 UTC m=+454.190759679 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/d61faf19-d78e-494d-928c-27a2ffbe33c0-cert") pod "odh-model-controller-7998c67dc4-mdqnr" (UID: "d61faf19-d78e-494d-928c-27a2ffbe33c0") : secret "odh-model-controller-webhook-cert" not found
Sep 17 14:12:32.703993 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:32.703964    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5p8v\" (UniqueName: \"kubernetes.io/projected/d61faf19-d78e-494d-928c-27a2ffbe33c0-kube-api-access-n5p8v\") pod \"odh-model-controller-7998c67dc4-mdqnr\" (UID: \"d61faf19-d78e-494d-928c-27a2ffbe33c0\") " pod="opendatahub/odh-model-controller-7998c67dc4-mdqnr"
Sep 17 14:12:32.747325 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:32.747277    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:12:32.888978 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:32.888902    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:12:33.145599 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:33.145522    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-hpmc6"]
Sep 17 14:12:33.148939 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:33.148923    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-hpmc6"
Sep 17 14:12:33.151199 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:33.151179    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 17 14:12:33.162587 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:33.162563    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-hpmc6"]
Sep 17 14:12:33.177403 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:33.177382    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:12:33.193528 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:33.193503    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/839e8ecf-313f-4dac-965a-9c071d3af679-tls-certs\") pod \"model-serving-api-54d89bd5fc-hpmc6\" (UID: \"839e8ecf-313f-4dac-965a-9c071d3af679\") " pod="opendatahub/model-serving-api-54d89bd5fc-hpmc6"
Sep 17 14:12:33.193678 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:33.193570    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d61faf19-d78e-494d-928c-27a2ffbe33c0-cert\") pod \"odh-model-controller-7998c67dc4-mdqnr\" (UID: \"d61faf19-d78e-494d-928c-27a2ffbe33c0\") " pod="opendatahub/odh-model-controller-7998c67dc4-mdqnr"
Sep 17 14:12:33.193678 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:33.193589    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xjmv\" (UniqueName: \"kubernetes.io/projected/839e8ecf-313f-4dac-965a-9c071d3af679-kube-api-access-6xjmv\") pod \"model-serving-api-54d89bd5fc-hpmc6\" (UID: \"839e8ecf-313f-4dac-965a-9c071d3af679\") " pod="opendatahub/model-serving-api-54d89bd5fc-hpmc6"
Sep 17 14:12:33.195963 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:33.195946    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d61faf19-d78e-494d-928c-27a2ffbe33c0-cert\") pod \"odh-model-controller-7998c67dc4-mdqnr\" (UID: \"d61faf19-d78e-494d-928c-27a2ffbe33c0\") " pod="opendatahub/odh-model-controller-7998c67dc4-mdqnr"
Sep 17 14:12:33.294127 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:33.294086    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xjmv\" (UniqueName: \"kubernetes.io/projected/839e8ecf-313f-4dac-965a-9c071d3af679-kube-api-access-6xjmv\") pod \"model-serving-api-54d89bd5fc-hpmc6\" (UID: \"839e8ecf-313f-4dac-965a-9c071d3af679\") " pod="opendatahub/model-serving-api-54d89bd5fc-hpmc6"
Sep 17 14:12:33.294328 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:33.294181    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/839e8ecf-313f-4dac-965a-9c071d3af679-tls-certs\") pod \"model-serving-api-54d89bd5fc-hpmc6\" (UID: \"839e8ecf-313f-4dac-965a-9c071d3af679\") " pod="opendatahub/model-serving-api-54d89bd5fc-hpmc6"
Sep 17 14:12:33.297066 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:33.297040    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/839e8ecf-313f-4dac-965a-9c071d3af679-tls-certs\") pod \"model-serving-api-54d89bd5fc-hpmc6\" (UID: \"839e8ecf-313f-4dac-965a-9c071d3af679\") " pod="opendatahub/model-serving-api-54d89bd5fc-hpmc6"
Sep 17 14:12:33.304299 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:33.304269    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xjmv\" (UniqueName: \"kubernetes.io/projected/839e8ecf-313f-4dac-965a-9c071d3af679-kube-api-access-6xjmv\") pod \"model-serving-api-54d89bd5fc-hpmc6\" (UID: \"839e8ecf-313f-4dac-965a-9c071d3af679\") " pod="opendatahub/model-serving-api-54d89bd5fc-hpmc6"
Sep 17 14:12:33.315532 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:33.315462    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:12:33.415968 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:33.415879    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-mdqnr"
Sep 17 14:12:33.459550 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:33.459522    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-hpmc6"
Sep 17 14:12:33.923392 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:33.923354    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-mdqnr"]
Sep 17 14:12:33.926839 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:33.926773    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-hpmc6"]
Sep 17 14:12:33.927622 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:12:33.927576    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd61faf19_d78e_494d_928c_27a2ffbe33c0.slice/crio-10fb818cc0131202c3a10193368de3c93d28dd8ff90721e0d486f0989bd9ef4a WatchSource:0}: Error finding container 10fb818cc0131202c3a10193368de3c93d28dd8ff90721e0d486f0989bd9ef4a: Status 404 returned error can't find the container with id 10fb818cc0131202c3a10193368de3c93d28dd8ff90721e0d486f0989bd9ef4a
Sep 17 14:12:33.929340 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:12:33.929305    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod839e8ecf_313f_4dac_965a_9c071d3af679.slice/crio-0f2d47df46feca41d7efea001c93a2fbd4d3476f8d9e14cbdec28dbac3153a2b WatchSource:0}: Error finding container 0f2d47df46feca41d7efea001c93a2fbd4d3476f8d9e14cbdec28dbac3153a2b: Status 404 returned error can't find the container with id 0f2d47df46feca41d7efea001c93a2fbd4d3476f8d9e14cbdec28dbac3153a2b
Sep 17 14:12:34.647640 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:34.647581    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-hpmc6" event={"ID":"839e8ecf-313f-4dac-965a-9c071d3af679","Type":"ContainerStarted","Data":"0f2d47df46feca41d7efea001c93a2fbd4d3476f8d9e14cbdec28dbac3153a2b"}
Sep 17 14:12:34.648880 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:34.648845    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-mdqnr" event={"ID":"d61faf19-d78e-494d-928c-27a2ffbe33c0","Type":"ContainerStarted","Data":"10fb818cc0131202c3a10193368de3c93d28dd8ff90721e0d486f0989bd9ef4a"}
Sep 17 14:12:35.247523 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:35.247476    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:12:35.247818 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:35.247559    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:12:35.247818 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:35.247594    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:12:35.658046 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:35.657925    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn" event={"ID":"d76458e2-4b38-4f8b-947a-839acc027b9e","Type":"ContainerStarted","Data":"3986b6f4ef8f091d2fe9cd5b8ccbb74d66116bdec476d68359e980675f4e412d"}
Sep 17 14:12:35.680762 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:35.679694    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn"
Sep 17 14:12:35.680762 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:35.680217    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn" podStartSLOduration=1.398877015 podStartE2EDuration="5.680202248s" totalImagesPullingTime="4.281325233s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:12:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:12:35.679643796 +0000 UTC m=+456.681187282" watchObservedRunningTime="2026-09-17 14:12:35.680202248 +0000 UTC m=+456.681745723"
Sep 17 14:12:36.286343 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:36.286296    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:12:36.388679 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:36.388632    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:12:36.685299 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:36.685272    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn"
Sep 17 14:12:37.668928 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:37.668891    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn"
Sep 17 14:12:37.670153 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:37.670129    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn"
Sep 17 14:12:39.680111 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:39.680069    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-mdqnr" event={"ID":"d61faf19-d78e-494d-928c-27a2ffbe33c0","Type":"ContainerStarted","Data":"031c69d7b00c2ad49d02f02f6f6ab1f9e85530adcc1e66a8badb89f35c4abbd2"}
Sep 17 14:12:39.680706 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:39.680168    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-mdqnr"
Sep 17 14:12:39.681868 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:39.681835    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-hpmc6" event={"ID":"839e8ecf-313f-4dac-965a-9c071d3af679","Type":"ContainerStarted","Data":"afda4890b63d5d188351ead9a9fd6a8ac0227adba6eb2a22eb49020e769f3741"}
Sep 17 14:12:39.682243 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:39.682217    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-hpmc6"
Sep 17 14:12:39.729391 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:39.729336    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-mdqnr" podStartSLOduration=2.978079426 podStartE2EDuration="7.729303842s" totalImagesPullingTime="4.751224416s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:12:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:12:39.727707544 +0000 UTC m=+460.729251018" watchObservedRunningTime="2026-09-17 14:12:39.729303842 +0000 UTC m=+460.730847307"
Sep 17 14:12:39.754238 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:39.754175    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-hpmc6" podStartSLOduration=2.057893941 podStartE2EDuration="6.754157431s" totalImagesPullingTime="4.69626349s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:12:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:12:39.75239095 +0000 UTC m=+460.753934425" watchObservedRunningTime="2026-09-17 14:12:39.754157431 +0000 UTC m=+460.755700905"
Sep 17 14:12:42.545319 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:42.545277    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:12:42.609652 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:42.609618    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-l5q8z"]
Sep 17 14:12:42.613144 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:42.613127    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-l5q8z"
Sep 17 14:12:42.616404 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:42.616385    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-v8ktb\""
Sep 17 14:12:42.616490 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:42.616410    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 17 14:12:42.628046 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:42.628017    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-l5q8z"]
Sep 17 14:12:42.651165 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:42.651127    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:12:42.691387 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:42.691360    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kctqp\" (UniqueName: \"kubernetes.io/projected/328fc0b9-c0f0-4e49-9756-2b88cb1f9576-kube-api-access-kctqp\") pod \"llmisvc-controller-manager-777c6bd7d5-l5q8z\" (UID: \"328fc0b9-c0f0-4e49-9756-2b88cb1f9576\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-l5q8z"
Sep 17 14:12:42.691534 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:42.691465    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/328fc0b9-c0f0-4e49-9756-2b88cb1f9576-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-l5q8z\" (UID: \"328fc0b9-c0f0-4e49-9756-2b88cb1f9576\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-l5q8z"
Sep 17 14:12:42.792024 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:42.791989    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/328fc0b9-c0f0-4e49-9756-2b88cb1f9576-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-l5q8z\" (UID: \"328fc0b9-c0f0-4e49-9756-2b88cb1f9576\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-l5q8z"
Sep 17 14:12:42.792207 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:42.792044    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kctqp\" (UniqueName: \"kubernetes.io/projected/328fc0b9-c0f0-4e49-9756-2b88cb1f9576-kube-api-access-kctqp\") pod \"llmisvc-controller-manager-777c6bd7d5-l5q8z\" (UID: \"328fc0b9-c0f0-4e49-9756-2b88cb1f9576\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-l5q8z"
Sep 17 14:12:42.792286 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:42.792265    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:12:42.794489 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:42.794453    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/328fc0b9-c0f0-4e49-9756-2b88cb1f9576-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-l5q8z\" (UID: \"328fc0b9-c0f0-4e49-9756-2b88cb1f9576\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-l5q8z"
Sep 17 14:12:42.810248 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:42.810181    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kctqp\" (UniqueName: \"kubernetes.io/projected/328fc0b9-c0f0-4e49-9756-2b88cb1f9576-kube-api-access-kctqp\") pod \"llmisvc-controller-manager-777c6bd7d5-l5q8z\" (UID: \"328fc0b9-c0f0-4e49-9756-2b88cb1f9576\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-l5q8z"
Sep 17 14:12:42.920182 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:42.920149    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:12:42.924100 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:42.924080    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-l5q8z"
Sep 17 14:12:43.189462 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:43.189373    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:12:43.216386 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:43.216342    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-l5q8z"]
Sep 17 14:12:43.218909 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:12:43.218881    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod328fc0b9_c0f0_4e49_9756_2b88cb1f9576.slice/crio-a0684f465e1ffbda468fbaa3f16d0dd095f6d692dac1bf8c8bf30318cde29a50 WatchSource:0}: Error finding container a0684f465e1ffbda468fbaa3f16d0dd095f6d692dac1bf8c8bf30318cde29a50: Status 404 returned error can't find the container with id a0684f465e1ffbda468fbaa3f16d0dd095f6d692dac1bf8c8bf30318cde29a50
Sep 17 14:12:43.701668 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:43.701636    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-l5q8z" event={"ID":"328fc0b9-c0f0-4e49-9756-2b88cb1f9576","Type":"ContainerStarted","Data":"a0684f465e1ffbda468fbaa3f16d0dd095f6d692dac1bf8c8bf30318cde29a50"}
Sep 17 14:12:47.725133 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:47.725095    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-l5q8z" event={"ID":"328fc0b9-c0f0-4e49-9756-2b88cb1f9576","Type":"ContainerStarted","Data":"c5fcdef237880500452fbfd6b7ccbacc552b9b6d0134ec11712b0dfa2b189d8b"}
Sep 17 14:12:47.725502 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:47.725225    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-l5q8z"
Sep 17 14:12:47.747300 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:47.747256    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-l5q8z" podStartSLOduration=1.873490629 podStartE2EDuration="5.747241406s" totalImagesPullingTime="3.873750777s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:12:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:12:47.745176122 +0000 UTC m=+468.746719595" watchObservedRunningTime="2026-09-17 14:12:47.747241406 +0000 UTC m=+468.748784942"
Sep 17 14:12:50.689167 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:50.689136    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-mdqnr"
Sep 17 14:12:51.695313 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:12:51.695278    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-hpmc6"
Sep 17 14:13:18.732831 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:13:18.732798    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-l5q8z"
Sep 17 14:13:25.802393 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:13:25.802353    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:13:33.572100 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:13:33.572064    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:13:33.679480 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:13:33.679443    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:13:51.269340 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:13:51.269306    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:13:54.073457 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:13:54.073378    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:13:54.441634 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:13:54.441534    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:13:54.861156 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:13:54.861115    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:13:57.047629 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:13:57.047579    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:13:57.903644 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:13:57.903591    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8r2f"]
Sep 17 14:13:57.907435 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:13:57.907410    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8r2f"
Sep 17 14:13:57.911518 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:13:57.911484    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 17 14:13:57.911518 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:13:57.911497    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 17 14:13:57.911720 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:13:57.911491    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-n6xsv\""
Sep 17 14:13:57.924514 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:13:57.924482    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8r2f"]
Sep 17 14:13:57.933999 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:13:57.933959    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:13:57.956834 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:13:57.956796    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6dzf\" (UniqueName: \"kubernetes.io/projected/24526ea7-ed61-48fb-be42-a433abd55b61-kube-api-access-j6dzf\") pod \"kuadrant-operator-controller-manager-76f8f686bf-x8r2f\" (UID: \"24526ea7-ed61-48fb-be42-a433abd55b61\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8r2f"
Sep 17 14:13:57.956975 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:13:57.956895    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/24526ea7-ed61-48fb-be42-a433abd55b61-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-x8r2f\" (UID: \"24526ea7-ed61-48fb-be42-a433abd55b61\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8r2f"
Sep 17 14:13:58.039464 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:13:58.039429    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:13:58.057960 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:13:58.057926    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/24526ea7-ed61-48fb-be42-a433abd55b61-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-x8r2f\" (UID: \"24526ea7-ed61-48fb-be42-a433abd55b61\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8r2f"
Sep 17 14:13:58.058334 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:13:58.057983    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6dzf\" (UniqueName: \"kubernetes.io/projected/24526ea7-ed61-48fb-be42-a433abd55b61-kube-api-access-j6dzf\") pod \"kuadrant-operator-controller-manager-76f8f686bf-x8r2f\" (UID: \"24526ea7-ed61-48fb-be42-a433abd55b61\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8r2f"
Sep 17 14:13:58.058334 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:13:58.058278    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/24526ea7-ed61-48fb-be42-a433abd55b61-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-x8r2f\" (UID: \"24526ea7-ed61-48fb-be42-a433abd55b61\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8r2f"
Sep 17 14:13:58.078652 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:13:58.078619    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6dzf\" (UniqueName: \"kubernetes.io/projected/24526ea7-ed61-48fb-be42-a433abd55b61-kube-api-access-j6dzf\") pod \"kuadrant-operator-controller-manager-76f8f686bf-x8r2f\" (UID: \"24526ea7-ed61-48fb-be42-a433abd55b61\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8r2f"
Sep 17 14:13:58.145365 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:13:58.145332    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:13:58.223980 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:13:58.223945    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8r2f"
Sep 17 14:13:58.505193 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:13:58.505140    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:13:58.505963 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:13:58.505936    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8r2f"]
Sep 17 14:13:58.508712 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:13:58.508684    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24526ea7_ed61_48fb_be42_a433abd55b61.slice/crio-5f913d327e96afe31a03077bb78e75a01fcb62b671eca4c438d26c3180805af4 WatchSource:0}: Error finding container 5f913d327e96afe31a03077bb78e75a01fcb62b671eca4c438d26c3180805af4: Status 404 returned error can't find the container with id 5f913d327e96afe31a03077bb78e75a01fcb62b671eca4c438d26c3180805af4
Sep 17 14:13:59.059759 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:13:59.059721    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8r2f" event={"ID":"24526ea7-ed61-48fb-be42-a433abd55b61","Type":"ContainerStarted","Data":"5f913d327e96afe31a03077bb78e75a01fcb62b671eca4c438d26c3180805af4"}
Sep 17 14:13:59.537095 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:13:59.537055    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:13:59.639190 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:13:59.639153    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:14:01.095817 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:01.095777    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:14:01.190103 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:01.190006    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:14:02.757164 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:02.757125    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-v7j87"]
Sep 17 14:14:02.761148 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:02.761123    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-v7j87"
Sep 17 14:14:02.763745 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:02.763715    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-27tsk\""
Sep 17 14:14:02.763873 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:02.763746    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 17 14:14:02.770581 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:02.770515    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-v7j87"]
Sep 17 14:14:02.802401 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:02.802357    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6qlf\" (UniqueName: \"kubernetes.io/projected/0a804f40-44d8-4775-8aa6-43cef6da55aa-kube-api-access-d6qlf\") pod \"dns-operator-controller-manager-7b84cd7c9c-v7j87\" (UID: \"0a804f40-44d8-4775-8aa6-43cef6da55aa\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-v7j87"
Sep 17 14:14:02.902990 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:02.902947    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6qlf\" (UniqueName: \"kubernetes.io/projected/0a804f40-44d8-4775-8aa6-43cef6da55aa-kube-api-access-d6qlf\") pod \"dns-operator-controller-manager-7b84cd7c9c-v7j87\" (UID: \"0a804f40-44d8-4775-8aa6-43cef6da55aa\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-v7j87"
Sep 17 14:14:02.911719 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:02.911669    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6qlf\" (UniqueName: \"kubernetes.io/projected/0a804f40-44d8-4775-8aa6-43cef6da55aa-kube-api-access-d6qlf\") pod \"dns-operator-controller-manager-7b84cd7c9c-v7j87\" (UID: \"0a804f40-44d8-4775-8aa6-43cef6da55aa\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-v7j87"
Sep 17 14:14:03.076413 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:03.076309    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-v7j87"
Sep 17 14:14:03.990671 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:03.990627    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-v7j87"]
Sep 17 14:14:03.992948 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:14:03.992919    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a804f40_44d8_4775_8aa6_43cef6da55aa.slice/crio-4003d629666b4ea8ee72a3df899c729d351e1fbfbe4966e7e6c8ef2845682a8d WatchSource:0}: Error finding container 4003d629666b4ea8ee72a3df899c729d351e1fbfbe4966e7e6c8ef2845682a8d: Status 404 returned error can't find the container with id 4003d629666b4ea8ee72a3df899c729d351e1fbfbe4966e7e6c8ef2845682a8d
Sep 17 14:14:04.089710 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:04.089670    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8r2f" event={"ID":"24526ea7-ed61-48fb-be42-a433abd55b61","Type":"ContainerStarted","Data":"a733830fcb6a0a05d6759bc3f9c4fb8f3d741eb78d754a64733305ec39a72c56"}
Sep 17 14:14:04.089922 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:04.089797    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8r2f"
Sep 17 14:14:04.091038 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:04.091015    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-v7j87" event={"ID":"0a804f40-44d8-4775-8aa6-43cef6da55aa","Type":"ContainerStarted","Data":"4003d629666b4ea8ee72a3df899c729d351e1fbfbe4966e7e6c8ef2845682a8d"}
Sep 17 14:14:04.091422 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:04.091403    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:14:04.113969 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:04.113910    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8r2f" podStartSLOduration=1.823433578 podStartE2EDuration="7.113892055s" totalImagesPullingTime="5.290458477s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:13:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:14:04.111952039 +0000 UTC m=+545.113495525" watchObservedRunningTime="2026-09-17 14:14:04.113892055 +0000 UTC m=+545.115435595"
Sep 17 14:14:04.259927 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:04.259840    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:14:06.103825 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:06.103785    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-v7j87" event={"ID":"0a804f40-44d8-4775-8aa6-43cef6da55aa","Type":"ContainerStarted","Data":"afe8fad576365854252b35412baedbd008d524350ca400dcc60b17a82034dfc5"}
Sep 17 14:14:06.104204 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:06.103862    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-v7j87"
Sep 17 14:14:06.129261 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:06.129202    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-v7j87" podStartSLOduration=2.209072177 podStartE2EDuration="4.129182398s" totalImagesPullingTime="1.920110221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:14:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:14:06.127667389 +0000 UTC m=+547.129210863" watchObservedRunningTime="2026-09-17 14:14:06.129182398 +0000 UTC m=+547.130725871"
Sep 17 14:14:06.725022 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:06.724981    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:14:07.138658 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:07.138546    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:14:07.857407 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:07.857371    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:14:07.957989 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:07.957948    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:14:08.057974 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:08.057937    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:14:08.161178 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:08.161091    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:14:08.280187 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:08.280155    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:14:08.357987 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:08.357948    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:14:15.099894 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:15.099862    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8r2f"
Sep 17 14:14:16.605279 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:16.605241    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:14:16.700601 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:16.700566    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:14:17.110869 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:17.110839    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-v7j87"
Sep 17 14:14:17.594910 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:17.594873    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8r2f"]
Sep 17 14:14:17.595093 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:17.595046    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8r2f" podUID="24526ea7-ed61-48fb-be42-a433abd55b61" containerName="manager" containerID="cri-o://a733830fcb6a0a05d6759bc3f9c4fb8f3d741eb78d754a64733305ec39a72c56" gracePeriod=2
Sep 17 14:14:17.605829 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:17.605801    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8r2f"]
Sep 17 14:14:17.609657 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:14:17.609589    2582 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"24526ea7-ed61-48fb-be42-a433abd55b61\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-76f8f686bf-x8r2f\": pods \"kuadrant-operator-controller-manager-76f8f686bf-x8r2f\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8r2f"
Sep 17 14:14:17.616880 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:17.616853    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rj88v"]
Sep 17 14:14:17.617257 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:17.617246    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24526ea7-ed61-48fb-be42-a433abd55b61" containerName="manager"
Sep 17 14:14:17.617303 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:17.617259    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="24526ea7-ed61-48fb-be42-a433abd55b61" containerName="manager"
Sep 17 14:14:17.617341 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:17.617332    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="24526ea7-ed61-48fb-be42-a433abd55b61" containerName="manager"
Sep 17 14:14:17.620475 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:17.620455    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rj88v"
Sep 17 14:14:17.622744 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:14:17.622720    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-x8r2f\" is forbidden: User \"system:node:ip-10-0-137-53.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-53.ec2.internal' and this object" podUID="24526ea7-ed61-48fb-be42-a433abd55b61" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8r2f"
Sep 17 14:14:17.636286 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:17.636249    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rj88v"]
Sep 17 14:14:17.744265 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:17.744231    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2b6q\" (UniqueName: \"kubernetes.io/projected/233009b0-0041-41dd-b296-c88281418371-kube-api-access-g2b6q\") pod \"kuadrant-operator-controller-manager-76f8f686bf-rj88v\" (UID: \"233009b0-0041-41dd-b296-c88281418371\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rj88v"
Sep 17 14:14:17.744442 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:17.744317    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/233009b0-0041-41dd-b296-c88281418371-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-rj88v\" (UID: \"233009b0-0041-41dd-b296-c88281418371\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rj88v"
Sep 17 14:14:17.841513 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:17.841480    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8r2f"
Sep 17 14:14:17.843791 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:14:17.843759    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-x8r2f\" is forbidden: User \"system:node:ip-10-0-137-53.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-53.ec2.internal' and this object" podUID="24526ea7-ed61-48fb-be42-a433abd55b61" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8r2f"
Sep 17 14:14:17.844852 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:17.844830    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/233009b0-0041-41dd-b296-c88281418371-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-rj88v\" (UID: \"233009b0-0041-41dd-b296-c88281418371\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rj88v"
Sep 17 14:14:17.844937 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:17.844904    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2b6q\" (UniqueName: \"kubernetes.io/projected/233009b0-0041-41dd-b296-c88281418371-kube-api-access-g2b6q\") pod \"kuadrant-operator-controller-manager-76f8f686bf-rj88v\" (UID: \"233009b0-0041-41dd-b296-c88281418371\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rj88v"
Sep 17 14:14:17.845155 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:17.845140    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/233009b0-0041-41dd-b296-c88281418371-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-rj88v\" (UID: \"233009b0-0041-41dd-b296-c88281418371\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rj88v"
Sep 17 14:14:17.860727 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:17.860703    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2b6q\" (UniqueName: \"kubernetes.io/projected/233009b0-0041-41dd-b296-c88281418371-kube-api-access-g2b6q\") pod \"kuadrant-operator-controller-manager-76f8f686bf-rj88v\" (UID: \"233009b0-0041-41dd-b296-c88281418371\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rj88v"
Sep 17 14:14:17.946062 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:17.946035    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/24526ea7-ed61-48fb-be42-a433abd55b61-extensions-socket-volume\") pod \"24526ea7-ed61-48fb-be42-a433abd55b61\" (UID: \"24526ea7-ed61-48fb-be42-a433abd55b61\") "
Sep 17 14:14:17.946223 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:17.946068    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j6dzf\" (UniqueName: \"kubernetes.io/projected/24526ea7-ed61-48fb-be42-a433abd55b61-kube-api-access-j6dzf\") pod \"24526ea7-ed61-48fb-be42-a433abd55b61\" (UID: \"24526ea7-ed61-48fb-be42-a433abd55b61\") "
Sep 17 14:14:17.946540 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:17.946516    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24526ea7-ed61-48fb-be42-a433abd55b61-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "24526ea7-ed61-48fb-be42-a433abd55b61" (UID: "24526ea7-ed61-48fb-be42-a433abd55b61"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:14:17.948123 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:17.948104    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24526ea7-ed61-48fb-be42-a433abd55b61-kube-api-access-j6dzf" (OuterVolumeSpecName: "kube-api-access-j6dzf") pod "24526ea7-ed61-48fb-be42-a433abd55b61" (UID: "24526ea7-ed61-48fb-be42-a433abd55b61"). InnerVolumeSpecName "kube-api-access-j6dzf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:14:17.969288 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:17.969258    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rj88v"
Sep 17 14:14:18.047360 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:18.047324    2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/24526ea7-ed61-48fb-be42-a433abd55b61-extensions-socket-volume\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:14:18.047360 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:18.047363    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6dzf\" (UniqueName: \"kubernetes.io/projected/24526ea7-ed61-48fb-be42-a433abd55b61-kube-api-access-j6dzf\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:14:18.168515 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:18.168429    2582 generic.go:356] "Generic (PLEG): container finished" podID="24526ea7-ed61-48fb-be42-a433abd55b61" containerID="a733830fcb6a0a05d6759bc3f9c4fb8f3d741eb78d754a64733305ec39a72c56" exitCode=0
Sep 17 14:14:18.168515 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:18.168477    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8r2f"
Sep 17 14:14:18.168736 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:18.168530    2582 scope.go:117] "RemoveContainer" containerID="a733830fcb6a0a05d6759bc3f9c4fb8f3d741eb78d754a64733305ec39a72c56"
Sep 17 14:14:18.170766 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:14:18.170738    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-x8r2f\" is forbidden: User \"system:node:ip-10-0-137-53.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-53.ec2.internal' and this object" podUID="24526ea7-ed61-48fb-be42-a433abd55b61" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8r2f"
Sep 17 14:14:18.177153 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:18.177134    2582 scope.go:117] "RemoveContainer" containerID="a733830fcb6a0a05d6759bc3f9c4fb8f3d741eb78d754a64733305ec39a72c56"
Sep 17 14:14:18.177412 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:14:18.177396    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a733830fcb6a0a05d6759bc3f9c4fb8f3d741eb78d754a64733305ec39a72c56\": container with ID starting with a733830fcb6a0a05d6759bc3f9c4fb8f3d741eb78d754a64733305ec39a72c56 not found: ID does not exist" containerID="a733830fcb6a0a05d6759bc3f9c4fb8f3d741eb78d754a64733305ec39a72c56"
Sep 17 14:14:18.177468 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:18.177419    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a733830fcb6a0a05d6759bc3f9c4fb8f3d741eb78d754a64733305ec39a72c56"} err="failed to get container status \"a733830fcb6a0a05d6759bc3f9c4fb8f3d741eb78d754a64733305ec39a72c56\": rpc error: code = NotFound desc = could not find container \"a733830fcb6a0a05d6759bc3f9c4fb8f3d741eb78d754a64733305ec39a72c56\": container with ID starting with a733830fcb6a0a05d6759bc3f9c4fb8f3d741eb78d754a64733305ec39a72c56 not found: ID does not exist"
Sep 17 14:14:18.178590 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:14:18.178566    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-x8r2f\" is forbidden: User \"system:node:ip-10-0-137-53.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-53.ec2.internal' and this object" podUID="24526ea7-ed61-48fb-be42-a433abd55b61" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8r2f"
Sep 17 14:14:18.247602 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:18.247573    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rj88v"]
Sep 17 14:14:18.250460 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:14:18.250425    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod233009b0_0041_41dd_b296_c88281418371.slice/crio-beb2586e380abb00d9027e4dc17235d81240417f15a4880ac1b7c90e5023b50b WatchSource:0}: Error finding container beb2586e380abb00d9027e4dc17235d81240417f15a4880ac1b7c90e5023b50b: Status 404 returned error can't find the container with id beb2586e380abb00d9027e4dc17235d81240417f15a4880ac1b7c90e5023b50b
Sep 17 14:14:18.358785 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:18.358747    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6lzpm"]
Sep 17 14:14:18.362459 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:18.362439    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6lzpm"
Sep 17 14:14:18.383561 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:18.383539    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6lzpm"]
Sep 17 14:14:18.402467 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:14:18.402438    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-x8r2f\" is forbidden: User \"system:node:ip-10-0-137-53.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-53.ec2.internal' and this object" podUID="24526ea7-ed61-48fb-be42-a433abd55b61" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8r2f"
Sep 17 14:14:18.452173 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:18.452138    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3a99f184-10e9-401d-b502-4bc6889b8d2b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-6lzpm\" (UID: \"3a99f184-10e9-401d-b502-4bc6889b8d2b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6lzpm"
Sep 17 14:14:18.452173 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:18.452174    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkszd\" (UniqueName: \"kubernetes.io/projected/3a99f184-10e9-401d-b502-4bc6889b8d2b-kube-api-access-nkszd\") pod \"kuadrant-operator-controller-manager-5769f99d9f-6lzpm\" (UID: \"3a99f184-10e9-401d-b502-4bc6889b8d2b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6lzpm"
Sep 17 14:14:18.553622 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:18.553574    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3a99f184-10e9-401d-b502-4bc6889b8d2b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-6lzpm\" (UID: \"3a99f184-10e9-401d-b502-4bc6889b8d2b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6lzpm"
Sep 17 14:14:18.553778 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:18.553631    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkszd\" (UniqueName: \"kubernetes.io/projected/3a99f184-10e9-401d-b502-4bc6889b8d2b-kube-api-access-nkszd\") pod \"kuadrant-operator-controller-manager-5769f99d9f-6lzpm\" (UID: \"3a99f184-10e9-401d-b502-4bc6889b8d2b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6lzpm"
Sep 17 14:14:18.553961 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:18.553941    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3a99f184-10e9-401d-b502-4bc6889b8d2b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-6lzpm\" (UID: \"3a99f184-10e9-401d-b502-4bc6889b8d2b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6lzpm"
Sep 17 14:14:18.561624 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:18.561579    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkszd\" (UniqueName: \"kubernetes.io/projected/3a99f184-10e9-401d-b502-4bc6889b8d2b-kube-api-access-nkszd\") pod \"kuadrant-operator-controller-manager-5769f99d9f-6lzpm\" (UID: \"3a99f184-10e9-401d-b502-4bc6889b8d2b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6lzpm"
Sep 17 14:14:18.672642 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:18.672590    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6lzpm"
Sep 17 14:14:18.958670 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:18.958635    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6lzpm"]
Sep 17 14:14:18.961996 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:14:18.961963    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a99f184_10e9_401d_b502_4bc6889b8d2b.slice/crio-b44258ae9b8d147806003ea892e03f7291fefffe6e9c3957f243b48233407950 WatchSource:0}: Error finding container b44258ae9b8d147806003ea892e03f7291fefffe6e9c3957f243b48233407950: Status 404 returned error can't find the container with id b44258ae9b8d147806003ea892e03f7291fefffe6e9c3957f243b48233407950
Sep 17 14:14:19.006254 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:19.006230    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:14:19.106774 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:19.106740    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:14:19.174913 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:19.174883    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6lzpm" event={"ID":"3a99f184-10e9-401d-b502-4bc6889b8d2b","Type":"ContainerStarted","Data":"cf3d4e8fd20829b33b290d1ea0b65bd462b4a52cac0c93f56c485d9caa3181fc"}
Sep 17 14:14:19.175086 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:19.174931    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6lzpm" event={"ID":"3a99f184-10e9-401d-b502-4bc6889b8d2b","Type":"ContainerStarted","Data":"b44258ae9b8d147806003ea892e03f7291fefffe6e9c3957f243b48233407950"}
Sep 17 14:14:19.175086 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:19.175004    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6lzpm"
Sep 17 14:14:19.176790 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:19.176765    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rj88v" event={"ID":"233009b0-0041-41dd-b296-c88281418371","Type":"ContainerStarted","Data":"c607e5400d3a86575b21854f7d3eaaee194a610ef81b1eaa8d9c9621706cda2a"}
Sep 17 14:14:19.176919 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:19.176805    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rj88v" event={"ID":"233009b0-0041-41dd-b296-c88281418371","Type":"ContainerStarted","Data":"beb2586e380abb00d9027e4dc17235d81240417f15a4880ac1b7c90e5023b50b"}
Sep 17 14:14:19.176919 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:19.176829    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rj88v"
Sep 17 14:14:19.196748 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:19.196699    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6lzpm" podStartSLOduration=1.196683905 podStartE2EDuration="1.196683905s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:14:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:14:19.194795478 +0000 UTC m=+560.196338951" watchObservedRunningTime="2026-09-17 14:14:19.196683905 +0000 UTC m=+560.198227380"
Sep 17 14:14:19.196884 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:14:19.196860    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-x8r2f\" is forbidden: User \"system:node:ip-10-0-137-53.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-53.ec2.internal' and this object" podUID="24526ea7-ed61-48fb-be42-a433abd55b61" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8r2f"
Sep 17 14:14:19.219702 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:19.219657    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rj88v" podStartSLOduration=2.219643247 podStartE2EDuration="2.219643247s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:14:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:14:19.219064401 +0000 UTC m=+560.220607874" watchObservedRunningTime="2026-09-17 14:14:19.219643247 +0000 UTC m=+560.221186721"
Sep 17 14:14:19.221133 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:14:19.221110    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-x8r2f\" is forbidden: User \"system:node:ip-10-0-137-53.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-53.ec2.internal' and this object" podUID="24526ea7-ed61-48fb-be42-a433abd55b61" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8r2f"
Sep 17 14:14:19.493004 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:14:19.492964    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-x8r2f\" is forbidden: User \"system:node:ip-10-0-137-53.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-53.ec2.internal' and this object" podUID="24526ea7-ed61-48fb-be42-a433abd55b61" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-x8r2f"
Sep 17 14:14:19.495311 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:19.495286    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24526ea7-ed61-48fb-be42-a433abd55b61" path="/var/lib/kubelet/pods/24526ea7-ed61-48fb-be42-a433abd55b61/volumes"
Sep 17 14:14:23.422456 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:23.422419    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:14:23.523392 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:23.523356    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:14:26.994767 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:26.994714    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:14:27.266531 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:27.266450    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:14:27.418655 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:27.418587    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:14:27.498256 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:27.498225    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:14:27.582510 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:27.582422    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:14:27.671030 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:27.670997    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:14:30.185047 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:30.185018    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rj88v"
Sep 17 14:14:30.185414 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:30.185155    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6lzpm"
Sep 17 14:14:30.266169 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:30.266133    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rj88v"]
Sep 17 14:14:30.266384 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:30.266359    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rj88v" podUID="233009b0-0041-41dd-b296-c88281418371" containerName="manager" containerID="cri-o://c607e5400d3a86575b21854f7d3eaaee194a610ef81b1eaa8d9c9621706cda2a" gracePeriod=10
Sep 17 14:14:30.525539 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:30.525515    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rj88v"
Sep 17 14:14:30.664266 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:30.664226    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/233009b0-0041-41dd-b296-c88281418371-extensions-socket-volume\") pod \"233009b0-0041-41dd-b296-c88281418371\" (UID: \"233009b0-0041-41dd-b296-c88281418371\") "
Sep 17 14:14:30.664438 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:30.664342    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g2b6q\" (UniqueName: \"kubernetes.io/projected/233009b0-0041-41dd-b296-c88281418371-kube-api-access-g2b6q\") pod \"233009b0-0041-41dd-b296-c88281418371\" (UID: \"233009b0-0041-41dd-b296-c88281418371\") "
Sep 17 14:14:30.664662 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:30.664601    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/233009b0-0041-41dd-b296-c88281418371-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "233009b0-0041-41dd-b296-c88281418371" (UID: "233009b0-0041-41dd-b296-c88281418371"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:14:30.666512 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:30.666480    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/233009b0-0041-41dd-b296-c88281418371-kube-api-access-g2b6q" (OuterVolumeSpecName: "kube-api-access-g2b6q") pod "233009b0-0041-41dd-b296-c88281418371" (UID: "233009b0-0041-41dd-b296-c88281418371"). InnerVolumeSpecName "kube-api-access-g2b6q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:14:30.765869 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:30.765836    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2b6q\" (UniqueName: \"kubernetes.io/projected/233009b0-0041-41dd-b296-c88281418371-kube-api-access-g2b6q\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:14:30.765869 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:30.765864    2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/233009b0-0041-41dd-b296-c88281418371-extensions-socket-volume\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:14:31.241014 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:31.240983    2582 generic.go:356] "Generic (PLEG): container finished" podID="233009b0-0041-41dd-b296-c88281418371" containerID="c607e5400d3a86575b21854f7d3eaaee194a610ef81b1eaa8d9c9621706cda2a" exitCode=0
Sep 17 14:14:31.241397 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:31.241038    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rj88v"
Sep 17 14:14:31.241397 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:31.241040    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rj88v" event={"ID":"233009b0-0041-41dd-b296-c88281418371","Type":"ContainerDied","Data":"c607e5400d3a86575b21854f7d3eaaee194a610ef81b1eaa8d9c9621706cda2a"}
Sep 17 14:14:31.241397 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:31.241176    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rj88v" event={"ID":"233009b0-0041-41dd-b296-c88281418371","Type":"ContainerDied","Data":"beb2586e380abb00d9027e4dc17235d81240417f15a4880ac1b7c90e5023b50b"}
Sep 17 14:14:31.241397 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:31.241198    2582 scope.go:117] "RemoveContainer" containerID="c607e5400d3a86575b21854f7d3eaaee194a610ef81b1eaa8d9c9621706cda2a"
Sep 17 14:14:31.250069 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:31.250051    2582 scope.go:117] "RemoveContainer" containerID="c607e5400d3a86575b21854f7d3eaaee194a610ef81b1eaa8d9c9621706cda2a"
Sep 17 14:14:31.250331 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:14:31.250309    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c607e5400d3a86575b21854f7d3eaaee194a610ef81b1eaa8d9c9621706cda2a\": container with ID starting with c607e5400d3a86575b21854f7d3eaaee194a610ef81b1eaa8d9c9621706cda2a not found: ID does not exist" containerID="c607e5400d3a86575b21854f7d3eaaee194a610ef81b1eaa8d9c9621706cda2a"
Sep 17 14:14:31.250379 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:31.250340    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c607e5400d3a86575b21854f7d3eaaee194a610ef81b1eaa8d9c9621706cda2a"} err="failed to get container status \"c607e5400d3a86575b21854f7d3eaaee194a610ef81b1eaa8d9c9621706cda2a\": rpc error: code = NotFound desc = could not find container \"c607e5400d3a86575b21854f7d3eaaee194a610ef81b1eaa8d9c9621706cda2a\": container with ID starting with c607e5400d3a86575b21854f7d3eaaee194a610ef81b1eaa8d9c9621706cda2a not found: ID does not exist"
Sep 17 14:14:31.263684 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:31.263657    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rj88v"]
Sep 17 14:14:31.267246 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:31.267226    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rj88v"]
Sep 17 14:14:31.338370 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:31.338331    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:14:31.460732 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:31.460698    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:14:31.493731 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:31.493666    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="233009b0-0041-41dd-b296-c88281418371" path="/var/lib/kubelet/pods/233009b0-0041-41dd-b296-c88281418371/volumes"
Sep 17 14:14:33.807253 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:33.807223    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:14:36.057597 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:36.057566    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:14:36.235332 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:36.235296    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:14:38.799085 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:38.799050    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:14:39.172519 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:39.172439    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:14:46.641446 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:46.641414    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb"]
Sep 17 14:14:46.641913 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:46.641806    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="233009b0-0041-41dd-b296-c88281418371" containerName="manager"
Sep 17 14:14:46.641913 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:46.641818    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="233009b0-0041-41dd-b296-c88281418371" containerName="manager"
Sep 17 14:14:46.641913 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:46.641889    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="233009b0-0041-41dd-b296-c88281418371" containerName="manager"
Sep 17 14:14:46.646721 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:46.646695    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb"
Sep 17 14:14:46.649837 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:46.649806    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-284z6\""
Sep 17 14:14:46.666355 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:46.666328    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb"]
Sep 17 14:14:46.710938 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:46.710905    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcwq2\" (UniqueName: \"kubernetes.io/projected/612ce2f4-8e66-49cd-ad34-acab04c857d2-kube-api-access-gcwq2\") pod \"maas-default-gateway-openshift-default-5ccb968c66-znmmb\" (UID: \"612ce2f4-8e66-49cd-ad34-acab04c857d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb"
Sep 17 14:14:46.711077 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:46.710958    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/612ce2f4-8e66-49cd-ad34-acab04c857d2-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-znmmb\" (UID: \"612ce2f4-8e66-49cd-ad34-acab04c857d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb"
Sep 17 14:14:46.711077 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:46.710988    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/612ce2f4-8e66-49cd-ad34-acab04c857d2-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-znmmb\" (UID: \"612ce2f4-8e66-49cd-ad34-acab04c857d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb"
Sep 17 14:14:46.711077 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:46.711017    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/612ce2f4-8e66-49cd-ad34-acab04c857d2-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-znmmb\" (UID: \"612ce2f4-8e66-49cd-ad34-acab04c857d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb"
Sep 17 14:14:46.711205 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:46.711103    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/612ce2f4-8e66-49cd-ad34-acab04c857d2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-znmmb\" (UID: \"612ce2f4-8e66-49cd-ad34-acab04c857d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb"
Sep 17 14:14:46.711205 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:46.711174    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/612ce2f4-8e66-49cd-ad34-acab04c857d2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-znmmb\" (UID: \"612ce2f4-8e66-49cd-ad34-acab04c857d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb"
Sep 17 14:14:46.711205 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:46.711198    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/612ce2f4-8e66-49cd-ad34-acab04c857d2-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-znmmb\" (UID: \"612ce2f4-8e66-49cd-ad34-acab04c857d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb"
Sep 17 14:14:46.711314 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:46.711231    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/612ce2f4-8e66-49cd-ad34-acab04c857d2-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-znmmb\" (UID: \"612ce2f4-8e66-49cd-ad34-acab04c857d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb"
Sep 17 14:14:46.711314 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:46.711252    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/612ce2f4-8e66-49cd-ad34-acab04c857d2-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-znmmb\" (UID: \"612ce2f4-8e66-49cd-ad34-acab04c857d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb"
Sep 17 14:14:46.712174 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:46.712148    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:14:46.812388 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:46.812353    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcwq2\" (UniqueName: \"kubernetes.io/projected/612ce2f4-8e66-49cd-ad34-acab04c857d2-kube-api-access-gcwq2\") pod \"maas-default-gateway-openshift-default-5ccb968c66-znmmb\" (UID: \"612ce2f4-8e66-49cd-ad34-acab04c857d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb"
Sep 17 14:14:46.812388 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:46.812396    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/612ce2f4-8e66-49cd-ad34-acab04c857d2-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-znmmb\" (UID: \"612ce2f4-8e66-49cd-ad34-acab04c857d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb"
Sep 17 14:14:46.812583 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:46.812413    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/612ce2f4-8e66-49cd-ad34-acab04c857d2-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-znmmb\" (UID: \"612ce2f4-8e66-49cd-ad34-acab04c857d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb"
Sep 17 14:14:46.812583 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:46.812543    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/612ce2f4-8e66-49cd-ad34-acab04c857d2-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-znmmb\" (UID: \"612ce2f4-8e66-49cd-ad34-acab04c857d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb"
Sep 17 14:14:46.812678 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:46.812664    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/612ce2f4-8e66-49cd-ad34-acab04c857d2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-znmmb\" (UID: \"612ce2f4-8e66-49cd-ad34-acab04c857d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb"
Sep 17 14:14:46.812741 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:46.812726    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/612ce2f4-8e66-49cd-ad34-acab04c857d2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-znmmb\" (UID: \"612ce2f4-8e66-49cd-ad34-acab04c857d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb"
Sep 17 14:14:46.812779 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:46.812763    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/612ce2f4-8e66-49cd-ad34-acab04c857d2-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-znmmb\" (UID: \"612ce2f4-8e66-49cd-ad34-acab04c857d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb"
Sep 17 14:14:46.812816 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:46.812795    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/612ce2f4-8e66-49cd-ad34-acab04c857d2-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-znmmb\" (UID: \"612ce2f4-8e66-49cd-ad34-acab04c857d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb"
Sep 17 14:14:46.812856 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:46.812811    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/612ce2f4-8e66-49cd-ad34-acab04c857d2-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-znmmb\" (UID: \"612ce2f4-8e66-49cd-ad34-acab04c857d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb"
Sep 17 14:14:46.812897 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:46.812861    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/612ce2f4-8e66-49cd-ad34-acab04c857d2-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-znmmb\" (UID: \"612ce2f4-8e66-49cd-ad34-acab04c857d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb"
Sep 17 14:14:46.812897 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:46.812877    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/612ce2f4-8e66-49cd-ad34-acab04c857d2-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-znmmb\" (UID: \"612ce2f4-8e66-49cd-ad34-acab04c857d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb"
Sep 17 14:14:46.813043 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:46.813028    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/612ce2f4-8e66-49cd-ad34-acab04c857d2-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-znmmb\" (UID: \"612ce2f4-8e66-49cd-ad34-acab04c857d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb"
Sep 17 14:14:46.813190 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:46.813166    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/612ce2f4-8e66-49cd-ad34-acab04c857d2-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-znmmb\" (UID: \"612ce2f4-8e66-49cd-ad34-acab04c857d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb"
Sep 17 14:14:46.813439 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:46.813412    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/612ce2f4-8e66-49cd-ad34-acab04c857d2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-znmmb\" (UID: \"612ce2f4-8e66-49cd-ad34-acab04c857d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb"
Sep 17 14:14:46.815139 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:46.815107    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/612ce2f4-8e66-49cd-ad34-acab04c857d2-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-znmmb\" (UID: \"612ce2f4-8e66-49cd-ad34-acab04c857d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb"
Sep 17 14:14:46.815262 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:46.815218    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/612ce2f4-8e66-49cd-ad34-acab04c857d2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-znmmb\" (UID: \"612ce2f4-8e66-49cd-ad34-acab04c857d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb"
Sep 17 14:14:46.822066 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:46.822043    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/612ce2f4-8e66-49cd-ad34-acab04c857d2-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-znmmb\" (UID: \"612ce2f4-8e66-49cd-ad34-acab04c857d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb"
Sep 17 14:14:46.822320 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:46.822299    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcwq2\" (UniqueName: \"kubernetes.io/projected/612ce2f4-8e66-49cd-ad34-acab04c857d2-kube-api-access-gcwq2\") pod \"maas-default-gateway-openshift-default-5ccb968c66-znmmb\" (UID: \"612ce2f4-8e66-49cd-ad34-acab04c857d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb"
Sep 17 14:14:46.958406 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:46.958381    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb"
Sep 17 14:14:47.251174 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:47.251109    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb"]
Sep 17 14:14:47.253581 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:14:47.253557    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod612ce2f4_8e66_49cd_ad34_acab04c857d2.slice/crio-118c08bfd1ff387ea38e6692bbe2b7ee517d06b71c4e68769f26aeb67c9c3047 WatchSource:0}: Error finding container 118c08bfd1ff387ea38e6692bbe2b7ee517d06b71c4e68769f26aeb67c9c3047: Status 404 returned error can't find the container with id 118c08bfd1ff387ea38e6692bbe2b7ee517d06b71c4e68769f26aeb67c9c3047
Sep 17 14:14:47.255494 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:47.255462    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:14:47.255644 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:47.255524    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:14:47.255644 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:47.255552    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:14:47.323857 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:47.323827    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb" event={"ID":"612ce2f4-8e66-49cd-ad34-acab04c857d2","Type":"ContainerStarted","Data":"1a5fc899419883b9322798570e933cabe0d34a9bbc671e11f6aaea858bffaaeb"}
Sep 17 14:14:47.324002 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:47.323889    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb" event={"ID":"612ce2f4-8e66-49cd-ad34-acab04c857d2","Type":"ContainerStarted","Data":"118c08bfd1ff387ea38e6692bbe2b7ee517d06b71c4e68769f26aeb67c9c3047"}
Sep 17 14:14:47.351388 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:47.351301    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:14:47.378748 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:47.378693    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb" podStartSLOduration=1.378675885 podStartE2EDuration="1.378675885s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:14:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:14:47.376489397 +0000 UTC m=+588.378032872" watchObservedRunningTime="2026-09-17 14:14:47.378675885 +0000 UTC m=+588.380219429"
Sep 17 14:14:47.958682 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:47.958653    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb"
Sep 17 14:14:48.525446 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:48.525406    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:14:48.625172 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:48.625140    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:14:48.724676 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:48.724646    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:14:48.964309 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:48.964281    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb"
Sep 17 14:14:49.313194 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:49.313118    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-lwqrg"]
Sep 17 14:14:49.315880 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:49.315863    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-lwqrg"
Sep 17 14:14:49.318746 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:49.318724    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-mfw9h\""
Sep 17 14:14:49.328340 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:49.328315    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-lwqrg"]
Sep 17 14:14:49.335249 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:49.335226    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb"
Sep 17 14:14:49.336130 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:49.336111    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb"
Sep 17 14:14:49.339546 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:49.339523    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bb869\" (UniqueName: \"kubernetes.io/projected/c7ed93c6-9213-4d43-b4dd-0b942c7ddfe0-kube-api-access-bb869\") pod \"authorino-7bcfd8f9cd-lwqrg\" (UID: \"c7ed93c6-9213-4d43-b4dd-0b942c7ddfe0\") " pod="kuadrant-system/authorino-7bcfd8f9cd-lwqrg"
Sep 17 14:14:49.383710 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:49.383668    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:14:49.440913 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:49.440879    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bb869\" (UniqueName: \"kubernetes.io/projected/c7ed93c6-9213-4d43-b4dd-0b942c7ddfe0-kube-api-access-bb869\") pod \"authorino-7bcfd8f9cd-lwqrg\" (UID: \"c7ed93c6-9213-4d43-b4dd-0b942c7ddfe0\") " pod="kuadrant-system/authorino-7bcfd8f9cd-lwqrg"
Sep 17 14:14:49.453247 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:49.453212    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bb869\" (UniqueName: \"kubernetes.io/projected/c7ed93c6-9213-4d43-b4dd-0b942c7ddfe0-kube-api-access-bb869\") pod \"authorino-7bcfd8f9cd-lwqrg\" (UID: \"c7ed93c6-9213-4d43-b4dd-0b942c7ddfe0\") " pod="kuadrant-system/authorino-7bcfd8f9cd-lwqrg"
Sep 17 14:14:49.519567 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:49.519533    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:14:49.626106 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:49.626033    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-lwqrg"
Sep 17 14:14:49.902047 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:49.901965    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-lwqrg"]
Sep 17 14:14:49.907376 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:14:49.907348    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc7ed93c6_9213_4d43_b4dd_0b942c7ddfe0.slice/crio-c8fdcdab9686041a370a59c80c62bde4eabfab19c1f6eb1edb0b921fb2ac2d6d WatchSource:0}: Error finding container c8fdcdab9686041a370a59c80c62bde4eabfab19c1f6eb1edb0b921fb2ac2d6d: Status 404 returned error can't find the container with id c8fdcdab9686041a370a59c80c62bde4eabfab19c1f6eb1edb0b921fb2ac2d6d
Sep 17 14:14:50.341021 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:50.340984    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-lwqrg" event={"ID":"c7ed93c6-9213-4d43-b4dd-0b942c7ddfe0","Type":"ContainerStarted","Data":"c8fdcdab9686041a370a59c80c62bde4eabfab19c1f6eb1edb0b921fb2ac2d6d"}
Sep 17 14:14:52.354654 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:52.354594    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-lwqrg" event={"ID":"c7ed93c6-9213-4d43-b4dd-0b942c7ddfe0","Type":"ContainerStarted","Data":"4c0d5c50621a92a5645012e01d67b0a58d3bba65f976be4eff1bb4b7036a3aad"}
Sep 17 14:14:52.389766 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:52.389711    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-lwqrg" podStartSLOduration=1.776208553 podStartE2EDuration="3.389696589s" totalImagesPullingTime="1.613488036s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:14:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:14:52.388764796 +0000 UTC m=+593.390308272" watchObservedRunningTime="2026-09-17 14:14:52.389696589 +0000 UTC m=+593.391240091"
Sep 17 14:14:52.439759 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:52.439721    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-lwqrg"]
Sep 17 14:14:54.365708 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:54.365667    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-lwqrg" podUID="c7ed93c6-9213-4d43-b4dd-0b942c7ddfe0" containerName="authorino" containerID="cri-o://4c0d5c50621a92a5645012e01d67b0a58d3bba65f976be4eff1bb4b7036a3aad" gracePeriod=30
Sep 17 14:14:54.608877 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:54.608853    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-lwqrg"
Sep 17 14:14:54.691732 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:54.691651    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bb869\" (UniqueName: \"kubernetes.io/projected/c7ed93c6-9213-4d43-b4dd-0b942c7ddfe0-kube-api-access-bb869\") pod \"c7ed93c6-9213-4d43-b4dd-0b942c7ddfe0\" (UID: \"c7ed93c6-9213-4d43-b4dd-0b942c7ddfe0\") "
Sep 17 14:14:54.693739 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:54.693702    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7ed93c6-9213-4d43-b4dd-0b942c7ddfe0-kube-api-access-bb869" (OuterVolumeSpecName: "kube-api-access-bb869") pod "c7ed93c6-9213-4d43-b4dd-0b942c7ddfe0" (UID: "c7ed93c6-9213-4d43-b4dd-0b942c7ddfe0"). InnerVolumeSpecName "kube-api-access-bb869". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:14:54.792432 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:54.792400    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bb869\" (UniqueName: \"kubernetes.io/projected/c7ed93c6-9213-4d43-b4dd-0b942c7ddfe0-kube-api-access-bb869\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:14:55.371620 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:55.371582    2582 generic.go:356] "Generic (PLEG): container finished" podID="c7ed93c6-9213-4d43-b4dd-0b942c7ddfe0" containerID="4c0d5c50621a92a5645012e01d67b0a58d3bba65f976be4eff1bb4b7036a3aad" exitCode=0
Sep 17 14:14:55.372093 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:55.371638    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-lwqrg"
Sep 17 14:14:55.372093 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:55.371640    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-lwqrg" event={"ID":"c7ed93c6-9213-4d43-b4dd-0b942c7ddfe0","Type":"ContainerDied","Data":"4c0d5c50621a92a5645012e01d67b0a58d3bba65f976be4eff1bb4b7036a3aad"}
Sep 17 14:14:55.372093 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:55.371696    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-lwqrg" event={"ID":"c7ed93c6-9213-4d43-b4dd-0b942c7ddfe0","Type":"ContainerDied","Data":"c8fdcdab9686041a370a59c80c62bde4eabfab19c1f6eb1edb0b921fb2ac2d6d"}
Sep 17 14:14:55.372093 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:55.371722    2582 scope.go:117] "RemoveContainer" containerID="4c0d5c50621a92a5645012e01d67b0a58d3bba65f976be4eff1bb4b7036a3aad"
Sep 17 14:14:55.380532 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:55.380517    2582 scope.go:117] "RemoveContainer" containerID="4c0d5c50621a92a5645012e01d67b0a58d3bba65f976be4eff1bb4b7036a3aad"
Sep 17 14:14:55.380780 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:14:55.380761    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c0d5c50621a92a5645012e01d67b0a58d3bba65f976be4eff1bb4b7036a3aad\": container with ID starting with 4c0d5c50621a92a5645012e01d67b0a58d3bba65f976be4eff1bb4b7036a3aad not found: ID does not exist" containerID="4c0d5c50621a92a5645012e01d67b0a58d3bba65f976be4eff1bb4b7036a3aad"
Sep 17 14:14:55.380831 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:55.380789    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c0d5c50621a92a5645012e01d67b0a58d3bba65f976be4eff1bb4b7036a3aad"} err="failed to get container status \"4c0d5c50621a92a5645012e01d67b0a58d3bba65f976be4eff1bb4b7036a3aad\": rpc error: code = NotFound desc = could not find container \"4c0d5c50621a92a5645012e01d67b0a58d3bba65f976be4eff1bb4b7036a3aad\": container with ID starting with 4c0d5c50621a92a5645012e01d67b0a58d3bba65f976be4eff1bb4b7036a3aad not found: ID does not exist"
Sep 17 14:14:55.397738 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:55.397718    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-lwqrg"]
Sep 17 14:14:55.404184 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:55.404162    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-lwqrg"]
Sep 17 14:14:55.493408 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:55.493381    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7ed93c6-9213-4d43-b4dd-0b942c7ddfe0" path="/var/lib/kubelet/pods/c7ed93c6-9213-4d43-b4dd-0b942c7ddfe0/volumes"
Sep 17 14:14:59.409564 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:59.409540    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-rr97k_45ffa13d-635a-46e8-8894-30902b9cf719/console-operator/2.log"
Sep 17 14:14:59.410077 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:14:59.409573    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-rr97k_45ffa13d-635a-46e8-8894-30902b9cf719/console-operator/2.log"
Sep 17 14:15:04.600436 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:04.600402    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:15:04.705110 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:04.705075    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:15:04.800868 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:04.800828    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:15:05.053044 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:05.053015    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-67kml"]
Sep 17 14:15:05.053397 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:05.053385    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7ed93c6-9213-4d43-b4dd-0b942c7ddfe0" containerName="authorino"
Sep 17 14:15:05.053441 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:05.053399    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7ed93c6-9213-4d43-b4dd-0b942c7ddfe0" containerName="authorino"
Sep 17 14:15:05.053487 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:05.053478    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7ed93c6-9213-4d43-b4dd-0b942c7ddfe0" containerName="authorino"
Sep 17 14:15:05.056654 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:05.056623    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-67kml"
Sep 17 14:15:05.059008 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:05.058975    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 17 14:15:05.059151 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:05.059097    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 17 14:15:05.059225 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:05.059204    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-9l2sj\""
Sep 17 14:15:05.059822 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:05.059802    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 17 14:15:05.066532 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:05.066507    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-67kml"]
Sep 17 14:15:05.107378 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:05.107325    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:15:05.196514 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:05.196475    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/8aaf43e4-c249-416a-b673-d16a0dc79e7d-data\") pod \"postgres-69fb4594cf-67kml\" (UID: \"8aaf43e4-c249-416a-b673-d16a0dc79e7d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-67kml"
Sep 17 14:15:05.196732 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:05.196561    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbtmq\" (UniqueName: \"kubernetes.io/projected/8aaf43e4-c249-416a-b673-d16a0dc79e7d-kube-api-access-dbtmq\") pod \"postgres-69fb4594cf-67kml\" (UID: \"8aaf43e4-c249-416a-b673-d16a0dc79e7d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-67kml"
Sep 17 14:15:05.297755 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:05.297711    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbtmq\" (UniqueName: \"kubernetes.io/projected/8aaf43e4-c249-416a-b673-d16a0dc79e7d-kube-api-access-dbtmq\") pod \"postgres-69fb4594cf-67kml\" (UID: \"8aaf43e4-c249-416a-b673-d16a0dc79e7d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-67kml"
Sep 17 14:15:05.297966 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:05.297798    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/8aaf43e4-c249-416a-b673-d16a0dc79e7d-data\") pod \"postgres-69fb4594cf-67kml\" (UID: \"8aaf43e4-c249-416a-b673-d16a0dc79e7d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-67kml"
Sep 17 14:15:05.298166 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:05.298144    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/8aaf43e4-c249-416a-b673-d16a0dc79e7d-data\") pod \"postgres-69fb4594cf-67kml\" (UID: \"8aaf43e4-c249-416a-b673-d16a0dc79e7d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-67kml"
Sep 17 14:15:05.306495 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:05.306426    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbtmq\" (UniqueName: \"kubernetes.io/projected/8aaf43e4-c249-416a-b673-d16a0dc79e7d-kube-api-access-dbtmq\") pod \"postgres-69fb4594cf-67kml\" (UID: \"8aaf43e4-c249-416a-b673-d16a0dc79e7d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-67kml"
Sep 17 14:15:05.368973 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:05.368945    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-67kml"
Sep 17 14:15:05.640723 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:05.640700    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-67kml"]
Sep 17 14:15:05.643312 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:15:05.643278    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8aaf43e4_c249_416a_b673_d16a0dc79e7d.slice/crio-b6032ccec126c57fdf961dce7f3a0c9eb7bc5eec728ad37e5c52362f57a7dfe0 WatchSource:0}: Error finding container b6032ccec126c57fdf961dce7f3a0c9eb7bc5eec728ad37e5c52362f57a7dfe0: Status 404 returned error can't find the container with id b6032ccec126c57fdf961dce7f3a0c9eb7bc5eec728ad37e5c52362f57a7dfe0
Sep 17 14:15:06.442418 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:06.442329    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-67kml" event={"ID":"8aaf43e4-c249-416a-b673-d16a0dc79e7d","Type":"ContainerStarted","Data":"b6032ccec126c57fdf961dce7f3a0c9eb7bc5eec728ad37e5c52362f57a7dfe0"}
Sep 17 14:15:10.468173 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:10.468127    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-67kml" event={"ID":"8aaf43e4-c249-416a-b673-d16a0dc79e7d","Type":"ContainerStarted","Data":"155f7ad19af18ad7b35f7c7a796692699bbaaf340d9d70ccd3f09e41c102b107"}
Sep 17 14:15:10.468651 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:10.468211    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-67kml"
Sep 17 14:15:10.486424 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:10.486375    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-67kml" podStartSLOduration=0.810177622 podStartE2EDuration="5.486360522s" totalImagesPullingTime="4.6761829s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:15:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:15:10.483955056 +0000 UTC m=+611.485498531" watchObservedRunningTime="2026-09-17 14:15:10.486360522 +0000 UTC m=+611.487903996"
Sep 17 14:15:16.502896 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:16.502864    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-67kml"
Sep 17 14:15:16.834929 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:16.834845    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:15:17.441336 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:17.441299    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7b4d9c8c74-cqf9r"]
Sep 17 14:15:17.445002 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:17.444983    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b4d9c8c74-cqf9r"
Sep 17 14:15:17.448317 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:17.448292    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 17 14:15:17.448445 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:17.448321    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-mfw9h\""
Sep 17 14:15:17.453248 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:17.453227    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7b4d9c8c74-cqf9r"]
Sep 17 14:15:17.522063 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:17.522029    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnq8v\" (UniqueName: \"kubernetes.io/projected/99963285-a9dd-4306-924d-c3533d421107-kube-api-access-cnq8v\") pod \"authorino-7b4d9c8c74-cqf9r\" (UID: \"99963285-a9dd-4306-924d-c3533d421107\") " pod="kuadrant-system/authorino-7b4d9c8c74-cqf9r"
Sep 17 14:15:17.522459 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:17.522087    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/99963285-a9dd-4306-924d-c3533d421107-tls-cert\") pod \"authorino-7b4d9c8c74-cqf9r\" (UID: \"99963285-a9dd-4306-924d-c3533d421107\") " pod="kuadrant-system/authorino-7b4d9c8c74-cqf9r"
Sep 17 14:15:17.623197 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:17.623158    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnq8v\" (UniqueName: \"kubernetes.io/projected/99963285-a9dd-4306-924d-c3533d421107-kube-api-access-cnq8v\") pod \"authorino-7b4d9c8c74-cqf9r\" (UID: \"99963285-a9dd-4306-924d-c3533d421107\") " pod="kuadrant-system/authorino-7b4d9c8c74-cqf9r"
Sep 17 14:15:17.623373 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:17.623249    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/99963285-a9dd-4306-924d-c3533d421107-tls-cert\") pod \"authorino-7b4d9c8c74-cqf9r\" (UID: \"99963285-a9dd-4306-924d-c3533d421107\") " pod="kuadrant-system/authorino-7b4d9c8c74-cqf9r"
Sep 17 14:15:17.625743 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:17.625713    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/99963285-a9dd-4306-924d-c3533d421107-tls-cert\") pod \"authorino-7b4d9c8c74-cqf9r\" (UID: \"99963285-a9dd-4306-924d-c3533d421107\") " pod="kuadrant-system/authorino-7b4d9c8c74-cqf9r"
Sep 17 14:15:17.631358 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:17.631333    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnq8v\" (UniqueName: \"kubernetes.io/projected/99963285-a9dd-4306-924d-c3533d421107-kube-api-access-cnq8v\") pod \"authorino-7b4d9c8c74-cqf9r\" (UID: \"99963285-a9dd-4306-924d-c3533d421107\") " pod="kuadrant-system/authorino-7b4d9c8c74-cqf9r"
Sep 17 14:15:17.755210 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:17.755179    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b4d9c8c74-cqf9r"
Sep 17 14:15:18.026269 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:18.026242    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7b4d9c8c74-cqf9r"]
Sep 17 14:15:18.028814 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:15:18.028786    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod99963285_a9dd_4306_924d_c3533d421107.slice/crio-b539ea69a951b44120cabb33d2a851e393b6a045a4b8cb1b8855954fbe8a9941 WatchSource:0}: Error finding container b539ea69a951b44120cabb33d2a851e393b6a045a4b8cb1b8855954fbe8a9941: Status 404 returned error can't find the container with id b539ea69a951b44120cabb33d2a851e393b6a045a4b8cb1b8855954fbe8a9941
Sep 17 14:15:18.159601 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:18.159568    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:15:18.513962 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:18.513931    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b4d9c8c74-cqf9r" event={"ID":"99963285-a9dd-4306-924d-c3533d421107","Type":"ContainerStarted","Data":"b539ea69a951b44120cabb33d2a851e393b6a045a4b8cb1b8855954fbe8a9941"}
Sep 17 14:15:19.520496 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:19.520463    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b4d9c8c74-cqf9r" event={"ID":"99963285-a9dd-4306-924d-c3533d421107","Type":"ContainerStarted","Data":"f9089034ce714ed1f3c95453595a8532bb539dd9e19c5c9def6aff6490d2be40"}
Sep 17 14:15:19.539907 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:19.539856    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7b4d9c8c74-cqf9r" podStartSLOduration=2.1146092 podStartE2EDuration="2.539840829s" totalImagesPullingTime="425.231629ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:15:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:15:19.538121797 +0000 UTC m=+620.539665270" watchObservedRunningTime="2026-09-17 14:15:19.539840829 +0000 UTC m=+620.541384303"
Sep 17 14:15:24.398095 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:24.398012    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:15:24.496916 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:24.496881    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:15:24.599743 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:24.599706    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:15:24.696631 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:24.696574    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:15:28.071181 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:28.071138    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:15:28.152004 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:28.151971    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:15:38.560021 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:38.559985    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:15:52.246684 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:52.246647    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:15:57.414453 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:57.414418    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:15:57.668694 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:57.668590    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:15:59.694290 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:15:59.694248    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:16:00.026638 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:00.026519    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:16:01.155047 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:01.155015    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:16:01.256131 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:01.256098    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:16:01.408203 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:01.408115    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:16:01.704339 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:01.704299    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:16:01.878059 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:01.878024    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-9x9j4"]
Sep 17 14:16:01.881691 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:01.881669    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9x9j4"
Sep 17 14:16:01.882011 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:01.881991    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:16:01.884531 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:01.884511    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-chv8d\""
Sep 17 14:16:01.885891 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:01.885869    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 17 14:16:01.899353 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:01.899328    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-9x9j4"]
Sep 17 14:16:01.968569 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:01.968419    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:16:02.013159 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:02.013119    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8afff3a4-0475-4e19-96a8-562397330290-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-9x9j4\" (UID: \"8afff3a4-0475-4e19-96a8-562397330290\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9x9j4"
Sep 17 14:16:02.013326 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:02.013194    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4b6j\" (UniqueName: \"kubernetes.io/projected/8afff3a4-0475-4e19-96a8-562397330290-kube-api-access-q4b6j\") pod \"payload-pre-processing-59fbc47bf8-9x9j4\" (UID: \"8afff3a4-0475-4e19-96a8-562397330290\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9x9j4"
Sep 17 14:16:02.114489 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:02.114452    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8afff3a4-0475-4e19-96a8-562397330290-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-9x9j4\" (UID: \"8afff3a4-0475-4e19-96a8-562397330290\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9x9j4"
Sep 17 14:16:02.114674 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:02.114540    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4b6j\" (UniqueName: \"kubernetes.io/projected/8afff3a4-0475-4e19-96a8-562397330290-kube-api-access-q4b6j\") pod \"payload-pre-processing-59fbc47bf8-9x9j4\" (UID: \"8afff3a4-0475-4e19-96a8-562397330290\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9x9j4"
Sep 17 14:16:02.115218 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:02.115199    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8afff3a4-0475-4e19-96a8-562397330290-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-9x9j4\" (UID: \"8afff3a4-0475-4e19-96a8-562397330290\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9x9j4"
Sep 17 14:16:02.122726 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:02.122696    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4b6j\" (UniqueName: \"kubernetes.io/projected/8afff3a4-0475-4e19-96a8-562397330290-kube-api-access-q4b6j\") pod \"payload-pre-processing-59fbc47bf8-9x9j4\" (UID: \"8afff3a4-0475-4e19-96a8-562397330290\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9x9j4"
Sep 17 14:16:02.192002 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:02.191967    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9x9j4"
Sep 17 14:16:02.479855 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:02.479827    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-9x9j4"]
Sep 17 14:16:02.481336 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:16:02.481309    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8afff3a4_0475_4e19_96a8_562397330290.slice/crio-beba67e634384e5e82bb4b9aa6d662eae08cf33d272ee7e109166117a6c29496 WatchSource:0}: Error finding container beba67e634384e5e82bb4b9aa6d662eae08cf33d272ee7e109166117a6c29496: Status 404 returned error can't find the container with id beba67e634384e5e82bb4b9aa6d662eae08cf33d272ee7e109166117a6c29496
Sep 17 14:16:02.742224 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:02.742115    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9x9j4" event={"ID":"8afff3a4-0475-4e19-96a8-562397330290","Type":"ContainerStarted","Data":"beba67e634384e5e82bb4b9aa6d662eae08cf33d272ee7e109166117a6c29496"}
Sep 17 14:16:04.762160 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:04.762126    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9x9j4" event={"ID":"8afff3a4-0475-4e19-96a8-562397330290","Type":"ContainerStarted","Data":"1f6cec893f2825584763b3ca31644cbb1e173dedb8be8e68c50514e9573f8c8c"}
Sep 17 14:16:04.762663 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:04.762260    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9x9j4"
Sep 17 14:16:04.784637 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:04.781000    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9x9j4" podStartSLOduration=2.1871629439999998 podStartE2EDuration="3.780974363s" totalImagesPullingTime="1.593811419s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:16:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:16:04.779202024 +0000 UTC m=+665.780745490" watchObservedRunningTime="2026-09-17 14:16:04.780974363 +0000 UTC m=+665.782517839"
Sep 17 14:16:11.801336 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:11.801302    2582 generic.go:356] "Generic (PLEG): container finished" podID="612ce2f4-8e66-49cd-ad34-acab04c857d2" containerID="1a5fc899419883b9322798570e933cabe0d34a9bbc671e11f6aaea858bffaaeb" exitCode=0
Sep 17 14:16:11.801809 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:11.801383    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb" event={"ID":"612ce2f4-8e66-49cd-ad34-acab04c857d2","Type":"ContainerDied","Data":"1a5fc899419883b9322798570e933cabe0d34a9bbc671e11f6aaea858bffaaeb"}
Sep 17 14:16:11.801860 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:11.801836    2582 scope.go:117] "RemoveContainer" containerID="1a5fc899419883b9322798570e933cabe0d34a9bbc671e11f6aaea858bffaaeb"
Sep 17 14:16:11.802819 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:11.802794    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:16:11.802872 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:11.802857    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:16:11.802913 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:11.802887    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:16:12.746374 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:12.746306    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:16:12.808588 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:12.808553    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb" event={"ID":"612ce2f4-8e66-49cd-ad34-acab04c857d2","Type":"ContainerStarted","Data":"44f11b4386325d03a6fa8b79e2c1e696b1bab8705b4b7e23a6607eeb39c83fc1"}
Sep 17 14:16:12.820730 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:12.820697    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:16:12.958707 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:12.958657    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb"
Sep 17 14:16:12.958951 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:12.958925    2582 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-znmmb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 17 14:16:12.959037 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:12.959002    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb" podUID="612ce2f4-8e66-49cd-ad34-acab04c857d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 17 14:16:13.959316 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:13.959283    2582 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-znmmb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 17 14:16:13.959722 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:13.959339    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb" podUID="612ce2f4-8e66-49cd-ad34-acab04c857d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 17 14:16:14.959485 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:14.959453    2582 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-znmmb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 17 14:16:14.959928 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:14.959505    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb" podUID="612ce2f4-8e66-49cd-ad34-acab04c857d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 17 14:16:15.770716 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:15.770683    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9x9j4"
Sep 17 14:16:15.959859 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:15.959822    2582 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-znmmb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 17 14:16:15.960373 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:15.959886    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb" podUID="612ce2f4-8e66-49cd-ad34-acab04c857d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 17 14:16:16.959207 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:16.959162    2582 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-znmmb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 17 14:16:16.959394 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:16.959224    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb" podUID="612ce2f4-8e66-49cd-ad34-acab04c857d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 17 14:16:17.959081 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:17.959041    2582 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-znmmb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 17 14:16:17.959478 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:17.959095    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb" podUID="612ce2f4-8e66-49cd-ad34-acab04c857d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 17 14:16:18.538347 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:18.538317    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:16:18.963411 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:18.963380    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb"
Sep 17 14:16:18.963802 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:18.963654    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb"
Sep 17 14:16:18.964310 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:18.964290    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-znmmb"
Sep 17 14:16:29.928792 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:29.928745    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:16:41.021269 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:41.021214    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:16:55.754193 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:16:55.754091    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:17:08.067110 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:17:08.067073    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:17:18.713508 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:17:18.713467    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:17:32.784335 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:17:32.784262    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:18:06.504301 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:06.504261    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:18:31.314398 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:31.314310    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-64c7cbf655-rsx9d"]
Sep 17 14:18:31.317669 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:31.317647    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-64c7cbf655-rsx9d"
Sep 17 14:18:31.326382 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:31.326355    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-64c7cbf655-rsx9d"]
Sep 17 14:18:31.437443 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:31.437405    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f403d577-cab9-41f5-b000-f8ae11ffabee-tls-cert\") pod \"authorino-64c7cbf655-rsx9d\" (UID: \"f403d577-cab9-41f5-b000-f8ae11ffabee\") " pod="kuadrant-system/authorino-64c7cbf655-rsx9d"
Sep 17 14:18:31.437649 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:31.437507    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6fqn\" (UniqueName: \"kubernetes.io/projected/f403d577-cab9-41f5-b000-f8ae11ffabee-kube-api-access-z6fqn\") pod \"authorino-64c7cbf655-rsx9d\" (UID: \"f403d577-cab9-41f5-b000-f8ae11ffabee\") " pod="kuadrant-system/authorino-64c7cbf655-rsx9d"
Sep 17 14:18:31.538203 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:31.538158    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6fqn\" (UniqueName: \"kubernetes.io/projected/f403d577-cab9-41f5-b000-f8ae11ffabee-kube-api-access-z6fqn\") pod \"authorino-64c7cbf655-rsx9d\" (UID: \"f403d577-cab9-41f5-b000-f8ae11ffabee\") " pod="kuadrant-system/authorino-64c7cbf655-rsx9d"
Sep 17 14:18:31.538407 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:31.538235    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f403d577-cab9-41f5-b000-f8ae11ffabee-tls-cert\") pod \"authorino-64c7cbf655-rsx9d\" (UID: \"f403d577-cab9-41f5-b000-f8ae11ffabee\") " pod="kuadrant-system/authorino-64c7cbf655-rsx9d"
Sep 17 14:18:31.540692 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:31.540662    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f403d577-cab9-41f5-b000-f8ae11ffabee-tls-cert\") pod \"authorino-64c7cbf655-rsx9d\" (UID: \"f403d577-cab9-41f5-b000-f8ae11ffabee\") " pod="kuadrant-system/authorino-64c7cbf655-rsx9d"
Sep 17 14:18:31.547131 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:31.547103    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6fqn\" (UniqueName: \"kubernetes.io/projected/f403d577-cab9-41f5-b000-f8ae11ffabee-kube-api-access-z6fqn\") pod \"authorino-64c7cbf655-rsx9d\" (UID: \"f403d577-cab9-41f5-b000-f8ae11ffabee\") " pod="kuadrant-system/authorino-64c7cbf655-rsx9d"
Sep 17 14:18:31.627923 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:31.627855    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-64c7cbf655-rsx9d"
Sep 17 14:18:31.898314 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:31.898258    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-64c7cbf655-rsx9d"]
Sep 17 14:18:31.900684 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:18:31.900659    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf403d577_cab9_41f5_b000_f8ae11ffabee.slice/crio-8d792b095061d43d097ed5e8dcce2bd2a357a570cd6bd56f01754ec8ca2b389b WatchSource:0}: Error finding container 8d792b095061d43d097ed5e8dcce2bd2a357a570cd6bd56f01754ec8ca2b389b: Status 404 returned error can't find the container with id 8d792b095061d43d097ed5e8dcce2bd2a357a570cd6bd56f01754ec8ca2b389b
Sep 17 14:18:31.902089 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:31.902068    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 14:18:32.556097 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:32.556060    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-64c7cbf655-rsx9d" event={"ID":"f403d577-cab9-41f5-b000-f8ae11ffabee","Type":"ContainerStarted","Data":"8d792b095061d43d097ed5e8dcce2bd2a357a570cd6bd56f01754ec8ca2b389b"}
Sep 17 14:18:33.564461 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:33.564423    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-64c7cbf655-rsx9d" event={"ID":"f403d577-cab9-41f5-b000-f8ae11ffabee","Type":"ContainerStarted","Data":"1e77e02ab1a1449f524c9b5e58b5e1c8d39ab9a8457acf7ef71345a30945b565"}
Sep 17 14:18:33.581582 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:33.581511    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-64c7cbf655-rsx9d" podStartSLOduration=1.8670139319999999 podStartE2EDuration="2.581489807s" totalImagesPullingTime="714.475875ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:18:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:18:33.58125954 +0000 UTC m=+814.582803017" watchObservedRunningTime="2026-09-17 14:18:33.581489807 +0000 UTC m=+814.583033273"
Sep 17 14:18:33.609290 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:33.609253    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7b4d9c8c74-cqf9r"]
Sep 17 14:18:33.609499 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:33.609472    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7b4d9c8c74-cqf9r" podUID="99963285-a9dd-4306-924d-c3533d421107" containerName="authorino" containerID="cri-o://f9089034ce714ed1f3c95453595a8532bb539dd9e19c5c9def6aff6490d2be40" gracePeriod=30
Sep 17 14:18:33.875261 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:33.875234    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b4d9c8c74-cqf9r"
Sep 17 14:18:33.965145 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:33.965103    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cnq8v\" (UniqueName: \"kubernetes.io/projected/99963285-a9dd-4306-924d-c3533d421107-kube-api-access-cnq8v\") pod \"99963285-a9dd-4306-924d-c3533d421107\" (UID: \"99963285-a9dd-4306-924d-c3533d421107\") "
Sep 17 14:18:33.965339 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:33.965171    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/99963285-a9dd-4306-924d-c3533d421107-tls-cert\") pod \"99963285-a9dd-4306-924d-c3533d421107\" (UID: \"99963285-a9dd-4306-924d-c3533d421107\") "
Sep 17 14:18:33.967299 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:33.967266    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99963285-a9dd-4306-924d-c3533d421107-kube-api-access-cnq8v" (OuterVolumeSpecName: "kube-api-access-cnq8v") pod "99963285-a9dd-4306-924d-c3533d421107" (UID: "99963285-a9dd-4306-924d-c3533d421107"). InnerVolumeSpecName "kube-api-access-cnq8v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:18:33.976393 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:33.976366    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99963285-a9dd-4306-924d-c3533d421107-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "99963285-a9dd-4306-924d-c3533d421107" (UID: "99963285-a9dd-4306-924d-c3533d421107"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:18:34.066246 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:34.066207    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cnq8v\" (UniqueName: \"kubernetes.io/projected/99963285-a9dd-4306-924d-c3533d421107-kube-api-access-cnq8v\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:18:34.066246 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:34.066243    2582 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/99963285-a9dd-4306-924d-c3533d421107-tls-cert\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:18:34.152311 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:34.152178    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:18:34.249123 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:34.249092    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:18:34.571265 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:34.571227    2582 generic.go:356] "Generic (PLEG): container finished" podID="99963285-a9dd-4306-924d-c3533d421107" containerID="f9089034ce714ed1f3c95453595a8532bb539dd9e19c5c9def6aff6490d2be40" exitCode=0
Sep 17 14:18:34.571696 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:34.571277    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b4d9c8c74-cqf9r"
Sep 17 14:18:34.571696 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:34.571316    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b4d9c8c74-cqf9r" event={"ID":"99963285-a9dd-4306-924d-c3533d421107","Type":"ContainerDied","Data":"f9089034ce714ed1f3c95453595a8532bb539dd9e19c5c9def6aff6490d2be40"}
Sep 17 14:18:34.571696 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:34.571361    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b4d9c8c74-cqf9r" event={"ID":"99963285-a9dd-4306-924d-c3533d421107","Type":"ContainerDied","Data":"b539ea69a951b44120cabb33d2a851e393b6a045a4b8cb1b8855954fbe8a9941"}
Sep 17 14:18:34.571696 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:34.571379    2582 scope.go:117] "RemoveContainer" containerID="f9089034ce714ed1f3c95453595a8532bb539dd9e19c5c9def6aff6490d2be40"
Sep 17 14:18:34.582285 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:34.582262    2582 scope.go:117] "RemoveContainer" containerID="f9089034ce714ed1f3c95453595a8532bb539dd9e19c5c9def6aff6490d2be40"
Sep 17 14:18:34.582567 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:18:34.582545    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9089034ce714ed1f3c95453595a8532bb539dd9e19c5c9def6aff6490d2be40\": container with ID starting with f9089034ce714ed1f3c95453595a8532bb539dd9e19c5c9def6aff6490d2be40 not found: ID does not exist" containerID="f9089034ce714ed1f3c95453595a8532bb539dd9e19c5c9def6aff6490d2be40"
Sep 17 14:18:34.582663 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:34.582576    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9089034ce714ed1f3c95453595a8532bb539dd9e19c5c9def6aff6490d2be40"} err="failed to get container status \"f9089034ce714ed1f3c95453595a8532bb539dd9e19c5c9def6aff6490d2be40\": rpc error: code = NotFound desc = could not find container \"f9089034ce714ed1f3c95453595a8532bb539dd9e19c5c9def6aff6490d2be40\": container with ID starting with f9089034ce714ed1f3c95453595a8532bb539dd9e19c5c9def6aff6490d2be40 not found: ID does not exist"
Sep 17 14:18:34.595806 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:34.595774    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7b4d9c8c74-cqf9r"]
Sep 17 14:18:34.599146 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:34.599123    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7b4d9c8c74-cqf9r"]
Sep 17 14:18:35.310770 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:35.310732    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:18:35.405157 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:35.405107    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:18:35.494391 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:35.494356    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99963285-a9dd-4306-924d-c3533d421107" path="/var/lib/kubelet/pods/99963285-a9dd-4306-924d-c3533d421107/volumes"
Sep 17 14:18:51.391175 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.391126    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:18:51.491298 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.491258    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:18:51.593356 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.593324    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:18:51.700419 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.700383    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md"]
Sep 17 14:18:51.701042 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.701020    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99963285-a9dd-4306-924d-c3533d421107" containerName="authorino"
Sep 17 14:18:51.701042 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.701040    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="99963285-a9dd-4306-924d-c3533d421107" containerName="authorino"
Sep 17 14:18:51.701193 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.701112    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="99963285-a9dd-4306-924d-c3533d421107" containerName="authorino"
Sep 17 14:18:51.705832 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.705811    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md"
Sep 17 14:18:51.708242 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.708222    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-1269b2-openshift-default-dockercfg-hwxrd\""
Sep 17 14:18:51.717046 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.717018    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md"]
Sep 17 14:18:51.735230 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.734887    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:18:51.768561 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.768528    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"]
Sep 17 14:18:51.772281 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.772257    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:18:51.774570 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.774548    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-fcd9f3-openshift-default-dockercfg-dw9tz\""
Sep 17 14:18:51.784270 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.784247    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"]
Sep 17 14:18:51.835452 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.835409    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-workload-socket\") pod \"e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md\" (UID: \"68dd2b1c-ed84-4291-8fb7-f61779fd5b85\") " pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md"
Sep 17 14:18:51.835646 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.835461    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-workload-certs\") pod \"e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md\" (UID: \"68dd2b1c-ed84-4291-8fb7-f61779fd5b85\") " pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md"
Sep 17 14:18:51.835646 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.835538    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-istio-podinfo\") pod \"e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md\" (UID: \"68dd2b1c-ed84-4291-8fb7-f61779fd5b85\") " pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md"
Sep 17 14:18:51.835646 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.835637    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-istio-token\") pod \"e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md\" (UID: \"68dd2b1c-ed84-4291-8fb7-f61779fd5b85\") " pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md"
Sep 17 14:18:51.835811 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.835690    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-istio-envoy\") pod \"e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md\" (UID: \"68dd2b1c-ed84-4291-8fb7-f61779fd5b85\") " pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md"
Sep 17 14:18:51.835811 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.835747    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmp7k\" (UniqueName: \"kubernetes.io/projected/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-kube-api-access-vmp7k\") pod \"e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md\" (UID: \"68dd2b1c-ed84-4291-8fb7-f61779fd5b85\") " pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md"
Sep 17 14:18:51.835811 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.835795    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-istio-data\") pod \"e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md\" (UID: \"68dd2b1c-ed84-4291-8fb7-f61779fd5b85\") " pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md"
Sep 17 14:18:51.835974 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.835825    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-istiod-ca-cert\") pod \"e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md\" (UID: \"68dd2b1c-ed84-4291-8fb7-f61779fd5b85\") " pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md"
Sep 17 14:18:51.835974 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.835871    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-credential-socket\") pod \"e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md\" (UID: \"68dd2b1c-ed84-4291-8fb7-f61779fd5b85\") " pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md"
Sep 17 14:18:51.842238 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.842198    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:18:51.848701 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.848670    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn"]
Sep 17 14:18:51.852760 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.852742    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn"
Sep 17 14:18:51.855754 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.855731    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-181b16-openshift-default-dockercfg-96glt\""
Sep 17 14:18:51.867764 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.867741    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn"]
Sep 17 14:18:51.935751 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.935718    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:18:51.936883 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.936853    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/725b0118-b60a-4972-a84a-d52af1642e0b-workload-socket\") pod \"e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn\" (UID: \"725b0118-b60a-4972-a84a-d52af1642e0b\") " pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn"
Sep 17 14:18:51.936883 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.936883    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/725b0118-b60a-4972-a84a-d52af1642e0b-istio-podinfo\") pod \"e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn\" (UID: \"725b0118-b60a-4972-a84a-d52af1642e0b\") " pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn"
Sep 17 14:18:51.937062 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.936909    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/368ca806-07df-48be-8ee6-3e8773114611-credential-socket\") pod \"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\" (UID: \"368ca806-07df-48be-8ee6-3e8773114611\") " pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:18:51.937062 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.936974    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-workload-socket\") pod \"e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md\" (UID: \"68dd2b1c-ed84-4291-8fb7-f61779fd5b85\") " pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md"
Sep 17 14:18:51.937062 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.936993    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-workload-certs\") pod \"e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md\" (UID: \"68dd2b1c-ed84-4291-8fb7-f61779fd5b85\") " pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md"
Sep 17 14:18:51.937062 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.937013    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/368ca806-07df-48be-8ee6-3e8773114611-workload-certs\") pod \"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\" (UID: \"368ca806-07df-48be-8ee6-3e8773114611\") " pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:18:51.937062 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.937031    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/368ca806-07df-48be-8ee6-3e8773114611-workload-socket\") pod \"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\" (UID: \"368ca806-07df-48be-8ee6-3e8773114611\") " pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:18:51.937062 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.937049    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8n5ww\" (UniqueName: \"kubernetes.io/projected/725b0118-b60a-4972-a84a-d52af1642e0b-kube-api-access-8n5ww\") pod \"e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn\" (UID: \"725b0118-b60a-4972-a84a-d52af1642e0b\") " pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn"
Sep 17 14:18:51.937304 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.937121    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-istio-podinfo\") pod \"e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md\" (UID: \"68dd2b1c-ed84-4291-8fb7-f61779fd5b85\") " pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md"
Sep 17 14:18:51.937304 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.937193    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/368ca806-07df-48be-8ee6-3e8773114611-istio-token\") pod \"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\" (UID: \"368ca806-07df-48be-8ee6-3e8773114611\") " pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:18:51.937304 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.937229    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/725b0118-b60a-4972-a84a-d52af1642e0b-workload-certs\") pod \"e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn\" (UID: \"725b0118-b60a-4972-a84a-d52af1642e0b\") " pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn"
Sep 17 14:18:51.937418 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.937313    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-istio-token\") pod \"e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md\" (UID: \"68dd2b1c-ed84-4291-8fb7-f61779fd5b85\") " pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md"
Sep 17 14:18:51.937418 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.937338    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/725b0118-b60a-4972-a84a-d52af1642e0b-istio-data\") pod \"e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn\" (UID: \"725b0118-b60a-4972-a84a-d52af1642e0b\") " pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn"
Sep 17 14:18:51.937418 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.937364    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/725b0118-b60a-4972-a84a-d52af1642e0b-istio-token\") pod \"e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn\" (UID: \"725b0118-b60a-4972-a84a-d52af1642e0b\") " pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn"
Sep 17 14:18:51.937418 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.937370    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-workload-certs\") pod \"e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md\" (UID: \"68dd2b1c-ed84-4291-8fb7-f61779fd5b85\") " pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md"
Sep 17 14:18:51.937418 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.937392    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/368ca806-07df-48be-8ee6-3e8773114611-istio-podinfo\") pod \"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\" (UID: \"368ca806-07df-48be-8ee6-3e8773114611\") " pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:18:51.937638 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.937421    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-workload-socket\") pod \"e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md\" (UID: \"68dd2b1c-ed84-4291-8fb7-f61779fd5b85\") " pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md"
Sep 17 14:18:51.937638 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.937452    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/368ca806-07df-48be-8ee6-3e8773114611-istiod-ca-cert\") pod \"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\" (UID: \"368ca806-07df-48be-8ee6-3e8773114611\") " pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:18:51.937638 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.937534    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-istio-envoy\") pod \"e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md\" (UID: \"68dd2b1c-ed84-4291-8fb7-f61779fd5b85\") " pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md"
Sep 17 14:18:51.937638 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.937568    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/725b0118-b60a-4972-a84a-d52af1642e0b-istiod-ca-cert\") pod \"e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn\" (UID: \"725b0118-b60a-4972-a84a-d52af1642e0b\") " pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn"
Sep 17 14:18:51.937638 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.937587    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmp7k\" (UniqueName: \"kubernetes.io/projected/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-kube-api-access-vmp7k\") pod \"e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md\" (UID: \"68dd2b1c-ed84-4291-8fb7-f61779fd5b85\") " pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md"
Sep 17 14:18:51.937638 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.937602    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/725b0118-b60a-4972-a84a-d52af1642e0b-istio-envoy\") pod \"e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn\" (UID: \"725b0118-b60a-4972-a84a-d52af1642e0b\") " pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn"
Sep 17 14:18:51.937638 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.937637    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-istio-data\") pod \"e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md\" (UID: \"68dd2b1c-ed84-4291-8fb7-f61779fd5b85\") " pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md"
Sep 17 14:18:51.938017 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.937663    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-istiod-ca-cert\") pod \"e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md\" (UID: \"68dd2b1c-ed84-4291-8fb7-f61779fd5b85\") " pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md"
Sep 17 14:18:51.938017 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.937699    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/368ca806-07df-48be-8ee6-3e8773114611-istio-envoy\") pod \"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\" (UID: \"368ca806-07df-48be-8ee6-3e8773114611\") " pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:18:51.938017 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.937723    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/725b0118-b60a-4972-a84a-d52af1642e0b-credential-socket\") pod \"e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn\" (UID: \"725b0118-b60a-4972-a84a-d52af1642e0b\") " pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn"
Sep 17 14:18:51.938017 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.937754    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-credential-socket\") pod \"e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md\" (UID: \"68dd2b1c-ed84-4291-8fb7-f61779fd5b85\") " pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md"
Sep 17 14:18:51.938017 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.937780    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/368ca806-07df-48be-8ee6-3e8773114611-istio-data\") pod \"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\" (UID: \"368ca806-07df-48be-8ee6-3e8773114611\") " pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:18:51.938017 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.937804    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jrjb\" (UniqueName: \"kubernetes.io/projected/368ca806-07df-48be-8ee6-3e8773114611-kube-api-access-7jrjb\") pod \"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\" (UID: \"368ca806-07df-48be-8ee6-3e8773114611\") " pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:18:51.938017 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.937924    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-istio-data\") pod \"e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md\" (UID: \"68dd2b1c-ed84-4291-8fb7-f61779fd5b85\") " pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md"
Sep 17 14:18:51.938277 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.938092    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-credential-socket\") pod \"e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md\" (UID: \"68dd2b1c-ed84-4291-8fb7-f61779fd5b85\") " pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md"
Sep 17 14:18:51.938409 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.938385    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-istiod-ca-cert\") pod \"e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md\" (UID: \"68dd2b1c-ed84-4291-8fb7-f61779fd5b85\") " pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md"
Sep 17 14:18:51.939856 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.939837    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-istio-podinfo\") pod \"e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md\" (UID: \"68dd2b1c-ed84-4291-8fb7-f61779fd5b85\") " pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md"
Sep 17 14:18:51.939947 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.939888    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-istio-envoy\") pod \"e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md\" (UID: \"68dd2b1c-ed84-4291-8fb7-f61779fd5b85\") " pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md"
Sep 17 14:18:51.945176 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.945155    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-istio-token\") pod \"e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md\" (UID: \"68dd2b1c-ed84-4291-8fb7-f61779fd5b85\") " pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md"
Sep 17 14:18:51.945255 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:51.945227    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmp7k\" (UniqueName: \"kubernetes.io/projected/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-kube-api-access-vmp7k\") pod \"e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md\" (UID: \"68dd2b1c-ed84-4291-8fb7-f61779fd5b85\") " pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md"
Sep 17 14:18:52.017458 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.017362    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md"
Sep 17 14:18:52.035013 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.034968    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:18:52.039070 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.039043    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/368ca806-07df-48be-8ee6-3e8773114611-workload-certs\") pod \"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\" (UID: \"368ca806-07df-48be-8ee6-3e8773114611\") " pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:18:52.039070 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.039081    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/368ca806-07df-48be-8ee6-3e8773114611-workload-socket\") pod \"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\" (UID: \"368ca806-07df-48be-8ee6-3e8773114611\") " pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:18:52.039277 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.039106    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8n5ww\" (UniqueName: \"kubernetes.io/projected/725b0118-b60a-4972-a84a-d52af1642e0b-kube-api-access-8n5ww\") pod \"e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn\" (UID: \"725b0118-b60a-4972-a84a-d52af1642e0b\") " pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn"
Sep 17 14:18:52.039277 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.039140    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/368ca806-07df-48be-8ee6-3e8773114611-istio-token\") pod \"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\" (UID: \"368ca806-07df-48be-8ee6-3e8773114611\") " pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:18:52.039277 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.039163    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/725b0118-b60a-4972-a84a-d52af1642e0b-workload-certs\") pod \"e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn\" (UID: \"725b0118-b60a-4972-a84a-d52af1642e0b\") " pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn"
Sep 17 14:18:52.039277 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.039215    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/725b0118-b60a-4972-a84a-d52af1642e0b-istio-data\") pod \"e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn\" (UID: \"725b0118-b60a-4972-a84a-d52af1642e0b\") " pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn"
Sep 17 14:18:52.039277 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.039243    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/725b0118-b60a-4972-a84a-d52af1642e0b-istio-token\") pod \"e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn\" (UID: \"725b0118-b60a-4972-a84a-d52af1642e0b\") " pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn"
Sep 17 14:18:52.039277 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.039266    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/368ca806-07df-48be-8ee6-3e8773114611-istio-podinfo\") pod \"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\" (UID: \"368ca806-07df-48be-8ee6-3e8773114611\") " pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:18:52.039645 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.039295    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/368ca806-07df-48be-8ee6-3e8773114611-istiod-ca-cert\") pod \"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\" (UID: \"368ca806-07df-48be-8ee6-3e8773114611\") " pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:18:52.039645 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.039347    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/725b0118-b60a-4972-a84a-d52af1642e0b-istiod-ca-cert\") pod \"e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn\" (UID: \"725b0118-b60a-4972-a84a-d52af1642e0b\") " pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn"
Sep 17 14:18:52.039645 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.039372    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/725b0118-b60a-4972-a84a-d52af1642e0b-istio-envoy\") pod \"e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn\" (UID: \"725b0118-b60a-4972-a84a-d52af1642e0b\") " pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn"
Sep 17 14:18:52.039645 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.039426    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/368ca806-07df-48be-8ee6-3e8773114611-istio-envoy\") pod \"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\" (UID: \"368ca806-07df-48be-8ee6-3e8773114611\") " pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:18:52.039645 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.039453    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/725b0118-b60a-4972-a84a-d52af1642e0b-credential-socket\") pod \"e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn\" (UID: \"725b0118-b60a-4972-a84a-d52af1642e0b\") " pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn"
Sep 17 14:18:52.039645 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.039486    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/368ca806-07df-48be-8ee6-3e8773114611-istio-data\") pod \"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\" (UID: \"368ca806-07df-48be-8ee6-3e8773114611\") " pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:18:52.039645 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.039511    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jrjb\" (UniqueName: \"kubernetes.io/projected/368ca806-07df-48be-8ee6-3e8773114611-kube-api-access-7jrjb\") pod \"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\" (UID: \"368ca806-07df-48be-8ee6-3e8773114611\") " pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:18:52.039645 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.039565    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/725b0118-b60a-4972-a84a-d52af1642e0b-workload-socket\") pod \"e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn\" (UID: \"725b0118-b60a-4972-a84a-d52af1642e0b\") " pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn"
Sep 17 14:18:52.039645 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.039591    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/725b0118-b60a-4972-a84a-d52af1642e0b-istio-podinfo\") pod \"e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn\" (UID: \"725b0118-b60a-4972-a84a-d52af1642e0b\") " pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn"
Sep 17 14:18:52.040644 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.039654    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/368ca806-07df-48be-8ee6-3e8773114611-credential-socket\") pod \"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\" (UID: \"368ca806-07df-48be-8ee6-3e8773114611\") " pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:18:52.040644 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.039727    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/368ca806-07df-48be-8ee6-3e8773114611-workload-socket\") pod \"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\" (UID: \"368ca806-07df-48be-8ee6-3e8773114611\") " pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:18:52.040644 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.039936    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/368ca806-07df-48be-8ee6-3e8773114611-workload-certs\") pod \"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\" (UID: \"368ca806-07df-48be-8ee6-3e8773114611\") " pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:18:52.040644 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.039984    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/368ca806-07df-48be-8ee6-3e8773114611-credential-socket\") pod \"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\" (UID: \"368ca806-07df-48be-8ee6-3e8773114611\") " pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:18:52.040644 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.040203    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/725b0118-b60a-4972-a84a-d52af1642e0b-workload-socket\") pod \"e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn\" (UID: \"725b0118-b60a-4972-a84a-d52af1642e0b\") " pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn"
Sep 17 14:18:52.040644 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.040283    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/725b0118-b60a-4972-a84a-d52af1642e0b-istio-data\") pod \"e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn\" (UID: \"725b0118-b60a-4972-a84a-d52af1642e0b\") " pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn"
Sep 17 14:18:52.041278 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.040745    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/725b0118-b60a-4972-a84a-d52af1642e0b-credential-socket\") pod \"e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn\" (UID: \"725b0118-b60a-4972-a84a-d52af1642e0b\") " pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn"
Sep 17 14:18:52.041278 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.040863    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/725b0118-b60a-4972-a84a-d52af1642e0b-istiod-ca-cert\") pod \"e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn\" (UID: \"725b0118-b60a-4972-a84a-d52af1642e0b\") " pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn"
Sep 17 14:18:52.041278 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.041088    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/725b0118-b60a-4972-a84a-d52af1642e0b-workload-certs\") pod \"e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn\" (UID: \"725b0118-b60a-4972-a84a-d52af1642e0b\") " pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn"
Sep 17 14:18:52.041278 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.041176    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/368ca806-07df-48be-8ee6-3e8773114611-istio-data\") pod \"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\" (UID: \"368ca806-07df-48be-8ee6-3e8773114611\") " pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:18:52.041854 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.041830    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/368ca806-07df-48be-8ee6-3e8773114611-istiod-ca-cert\") pod \"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\" (UID: \"368ca806-07df-48be-8ee6-3e8773114611\") " pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:18:52.044198 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.044173    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/725b0118-b60a-4972-a84a-d52af1642e0b-istio-envoy\") pod \"e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn\" (UID: \"725b0118-b60a-4972-a84a-d52af1642e0b\") " pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn"
Sep 17 14:18:52.044367 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.044340    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/368ca806-07df-48be-8ee6-3e8773114611-istio-podinfo\") pod \"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\" (UID: \"368ca806-07df-48be-8ee6-3e8773114611\") " pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:18:52.045105 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.045084    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/725b0118-b60a-4972-a84a-d52af1642e0b-istio-podinfo\") pod \"e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn\" (UID: \"725b0118-b60a-4972-a84a-d52af1642e0b\") " pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn"
Sep 17 14:18:52.045468 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.045437    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/368ca806-07df-48be-8ee6-3e8773114611-istio-envoy\") pod \"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\" (UID: \"368ca806-07df-48be-8ee6-3e8773114611\") " pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:18:52.048699 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.048642    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/725b0118-b60a-4972-a84a-d52af1642e0b-istio-token\") pod \"e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn\" (UID: \"725b0118-b60a-4972-a84a-d52af1642e0b\") " pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn"
Sep 17 14:18:52.048699 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.048648    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jrjb\" (UniqueName: \"kubernetes.io/projected/368ca806-07df-48be-8ee6-3e8773114611-kube-api-access-7jrjb\") pod \"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\" (UID: \"368ca806-07df-48be-8ee6-3e8773114611\") " pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:18:52.049042 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.049018    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/368ca806-07df-48be-8ee6-3e8773114611-istio-token\") pod \"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\" (UID: \"368ca806-07df-48be-8ee6-3e8773114611\") " pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:18:52.049307 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.049283    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8n5ww\" (UniqueName: \"kubernetes.io/projected/725b0118-b60a-4972-a84a-d52af1642e0b-kube-api-access-8n5ww\") pod \"e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn\" (UID: \"725b0118-b60a-4972-a84a-d52af1642e0b\") " pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn"
Sep 17 14:18:52.085255 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.085088    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:18:52.164200 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.164144    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn"
Sep 17 14:18:52.308907 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.308876    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md"]
Sep 17 14:18:52.310185 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:18:52.310163    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68dd2b1c_ed84_4291_8fb7_f61779fd5b85.slice/crio-4af29a761f2bd4c594e1d5f38d34530762379f227551e2c851af1bb00b1578b3 WatchSource:0}: Error finding container 4af29a761f2bd4c594e1d5f38d34530762379f227551e2c851af1bb00b1578b3: Status 404 returned error can't find the container with id 4af29a761f2bd4c594e1d5f38d34530762379f227551e2c851af1bb00b1578b3
Sep 17 14:18:52.312421 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.312388    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:18:52.312511 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.312456    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:18:52.312511 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.312484    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:18:52.370965 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.370943    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"]
Sep 17 14:18:52.373360 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:18:52.373334    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod368ca806_07df_48be_8ee6_3e8773114611.slice/crio-d15ed72e9fb289407f344e7dd09ecd4a0441a82d585b4e8ff114b87593970f33 WatchSource:0}: Error finding container d15ed72e9fb289407f344e7dd09ecd4a0441a82d585b4e8ff114b87593970f33: Status 404 returned error can't find the container with id d15ed72e9fb289407f344e7dd09ecd4a0441a82d585b4e8ff114b87593970f33
Sep 17 14:18:52.375908 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.375875    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:18:52.375974 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.375954    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:18:52.376011 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.375990    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:18:52.444064 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.444040    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn"]
Sep 17 14:18:52.446670 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:18:52.446642    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod725b0118_b60a_4972_a84a_d52af1642e0b.slice/crio-685840ed8f4e1929e668466be4a5b823c17a572ce9c4eb2e37312aed153d0cc0 WatchSource:0}: Error finding container 685840ed8f4e1929e668466be4a5b823c17a572ce9c4eb2e37312aed153d0cc0: Status 404 returned error can't find the container with id 685840ed8f4e1929e668466be4a5b823c17a572ce9c4eb2e37312aed153d0cc0
Sep 17 14:18:52.449124 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.449088    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:18:52.449232 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.449156    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:18:52.449232 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.449188    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:18:52.670831 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.670663    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md" event={"ID":"68dd2b1c-ed84-4291-8fb7-f61779fd5b85","Type":"ContainerStarted","Data":"ed106bce24d830f2308cc17efa0004d937f823952dc96ce55a4beb200a5caf86"}
Sep 17 14:18:52.670831 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.670731    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md" event={"ID":"68dd2b1c-ed84-4291-8fb7-f61779fd5b85","Type":"ContainerStarted","Data":"4af29a761f2bd4c594e1d5f38d34530762379f227551e2c851af1bb00b1578b3"}
Sep 17 14:18:52.673765 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.673734    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn" event={"ID":"725b0118-b60a-4972-a84a-d52af1642e0b","Type":"ContainerStarted","Data":"a7dfc2b887f2083f84aa4641efa503bcf0a4325a9468bdfd34fd53fda29ef71b"}
Sep 17 14:18:52.673915 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.673789    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn" event={"ID":"725b0118-b60a-4972-a84a-d52af1642e0b","Type":"ContainerStarted","Data":"685840ed8f4e1929e668466be4a5b823c17a572ce9c4eb2e37312aed153d0cc0"}
Sep 17 14:18:52.676563 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.675949    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6" event={"ID":"368ca806-07df-48be-8ee6-3e8773114611","Type":"ContainerStarted","Data":"7000bfc51c007aaa54be164e8b608517d22220b1155dd9ec6af3c4fe1501b0c9"}
Sep 17 14:18:52.676563 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.675997    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6" event={"ID":"368ca806-07df-48be-8ee6-3e8773114611","Type":"ContainerStarted","Data":"d15ed72e9fb289407f344e7dd09ecd4a0441a82d585b4e8ff114b87593970f33"}
Sep 17 14:18:52.695962 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.695916    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md" podStartSLOduration=1.695900005 podStartE2EDuration="1.695900005s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:18:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:18:52.694357939 +0000 UTC m=+833.695901414" watchObservedRunningTime="2026-09-17 14:18:52.695900005 +0000 UTC m=+833.697443478"
Sep 17 14:18:52.716948 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.716890    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6" podStartSLOduration=1.7168686960000001 podStartE2EDuration="1.716868696s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:18:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:18:52.713244701 +0000 UTC m=+833.714788186" watchObservedRunningTime="2026-09-17 14:18:52.716868696 +0000 UTC m=+833.718412172"
Sep 17 14:18:52.737981 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.737916    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn" podStartSLOduration=1.737894994 podStartE2EDuration="1.737894994s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:18:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:18:52.736936092 +0000 UTC m=+833.738479568" watchObservedRunningTime="2026-09-17 14:18:52.737894994 +0000 UTC m=+833.739438469"
Sep 17 14:18:52.788603 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.788568    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:18:52.896997 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:52.896953    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:18:53.018009 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:53.017974    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md"
Sep 17 14:18:53.024205 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:53.024176    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md"
Sep 17 14:18:53.086049 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:53.086013    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:18:53.091059 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:53.091030    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:18:53.164312 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:53.164275    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn"
Sep 17 14:18:53.168976 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:53.168950    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn"
Sep 17 14:18:53.280640 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:53.280532    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:18:53.429283 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:53.429248    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7877-e34dd9"]
Sep 17 14:18:53.433443 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:53.433420    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7877-e34dd9"
Sep 17 14:18:53.436269 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:53.436200    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-25mz9\""
Sep 17 14:18:53.438116 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:53.438095    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7877-e34dd9"]
Sep 17 14:18:53.556213 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:53.556124    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6fvx\" (UniqueName: \"kubernetes.io/projected/65ccf32c-3fea-47ff-a18b-65fe74b0ca3c-kube-api-access-z6fvx\") pod \"test-curl-7877-e34dd9\" (UID: \"65ccf32c-3fea-47ff-a18b-65fe74b0ca3c\") " pod="opendatahub/test-curl-7877-e34dd9"
Sep 17 14:18:53.657713 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:53.657672    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6fvx\" (UniqueName: \"kubernetes.io/projected/65ccf32c-3fea-47ff-a18b-65fe74b0ca3c-kube-api-access-z6fvx\") pod \"test-curl-7877-e34dd9\" (UID: \"65ccf32c-3fea-47ff-a18b-65fe74b0ca3c\") " pod="opendatahub/test-curl-7877-e34dd9"
Sep 17 14:18:53.667074 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:53.667041    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6fvx\" (UniqueName: \"kubernetes.io/projected/65ccf32c-3fea-47ff-a18b-65fe74b0ca3c-kube-api-access-z6fvx\") pod \"test-curl-7877-e34dd9\" (UID: \"65ccf32c-3fea-47ff-a18b-65fe74b0ca3c\") " pod="opendatahub/test-curl-7877-e34dd9"
Sep 17 14:18:53.682112 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:53.682086    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:18:53.682266 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:53.682151    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn"
Sep 17 14:18:53.682266 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:53.682181    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md"
Sep 17 14:18:53.682796 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:53.682755    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md"
Sep 17 14:18:53.682918 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:53.682908    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:18:53.682962 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:53.682951    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn"
Sep 17 14:18:53.755192 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:53.755148    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7877-e34dd9"
Sep 17 14:18:54.029386 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:54.029350    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7877-e34dd9"]
Sep 17 14:18:54.030985 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:18:54.030948    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod65ccf32c_3fea_47ff_a18b_65fe74b0ca3c.slice/crio-a6717e3dc561c60d517b1ead54cadb9e219fefd7816abba11fa2bd7ed96a5ff2 WatchSource:0}: Error finding container a6717e3dc561c60d517b1ead54cadb9e219fefd7816abba11fa2bd7ed96a5ff2: Status 404 returned error can't find the container with id a6717e3dc561c60d517b1ead54cadb9e219fefd7816abba11fa2bd7ed96a5ff2
Sep 17 14:18:54.691391 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:54.691355    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7877-e34dd9" event={"ID":"65ccf32c-3fea-47ff-a18b-65fe74b0ca3c","Type":"ContainerStarted","Data":"a6717e3dc561c60d517b1ead54cadb9e219fefd7816abba11fa2bd7ed96a5ff2"}
Sep 17 14:18:55.573891 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:55.573860    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:18:55.698356 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:55.698316    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7877-e34dd9" event={"ID":"65ccf32c-3fea-47ff-a18b-65fe74b0ca3c","Type":"ContainerStarted","Data":"45929a9a93389f7661fb92d05cd6cf1c81b4abe30e34cb42e9d6b4f3f3f02dd3"}
Sep 17 14:18:55.713720 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:55.713665    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7877-e34dd9" podStartSLOduration=1.874462216 podStartE2EDuration="2.713648873s" totalImagesPullingTime="839.186657ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:18:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:18:55.713333453 +0000 UTC m=+836.714876928" watchObservedRunningTime="2026-09-17 14:18:55.713648873 +0000 UTC m=+836.715192346"
Sep 17 14:18:56.305798 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:56.305758    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7877-e34dd9"]
Sep 17 14:18:56.309432 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:56.309374    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7877-e34dd9"]
Sep 17 14:18:56.332965 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:56.332926    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:18:56.604589 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:56.604502    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:18:56.694912 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:56.694873    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:18:56.703754 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:18:56.703730    2582 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7877-e34dd9" container="test-curl-7877-e34dd9"
Sep 17 14:18:57.049642 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:57.049592    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:18:57.157573 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:57.157395    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:18:57.241742 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:57.241707    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:18:57.267766 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:57.267732    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-1269b2-fdd4895d7-mr8fh"]
Sep 17 14:18:57.268444 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:57.268420    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65ccf32c-3fea-47ff-a18b-65fe74b0ca3c" containerName="test-curl-7877-e34dd9"
Sep 17 14:18:57.268561 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:57.268446    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="65ccf32c-3fea-47ff-a18b-65fe74b0ca3c" containerName="test-curl-7877-e34dd9"
Sep 17 14:18:57.268675 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:57.268577    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="65ccf32c-3fea-47ff-a18b-65fe74b0ca3c" containerName="test-curl-7877-e34dd9"
Sep 17 14:18:57.283518 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:57.283471    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-1269b2-fdd4895d7-mr8fh"
Sep 17 14:18:57.286724 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:57.286699    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-1269b2\""
Sep 17 14:18:57.286881 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:57.286742    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-1269b2-dockercfg-595h8\""
Sep 17 14:18:57.288349 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:57.288323    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-1269b2-fdd4895d7-mr8fh"]
Sep 17 14:18:57.293908 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:57.293774    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a17e2253-de02-4951-b5f4-e6eff7611b15-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-1269b2-fdd4895d7-mr8fh\" (UID: \"a17e2253-de02-4951-b5f4-e6eff7611b15\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-1269b2-fdd4895d7-mr8fh"
Sep 17 14:18:57.293983 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:57.293915    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8pc4\" (UniqueName: \"kubernetes.io/projected/a17e2253-de02-4951-b5f4-e6eff7611b15-kube-api-access-d8pc4\") pod \"payload-processing-e2e-worker-w0-1269b2-fdd4895d7-mr8fh\" (UID: \"a17e2253-de02-4951-b5f4-e6eff7611b15\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-1269b2-fdd4895d7-mr8fh"
Sep 17 14:18:57.394841 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:57.394759    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8pc4\" (UniqueName: \"kubernetes.io/projected/a17e2253-de02-4951-b5f4-e6eff7611b15-kube-api-access-d8pc4\") pod \"payload-processing-e2e-worker-w0-1269b2-fdd4895d7-mr8fh\" (UID: \"a17e2253-de02-4951-b5f4-e6eff7611b15\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-1269b2-fdd4895d7-mr8fh"
Sep 17 14:18:57.394841 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:57.394825    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a17e2253-de02-4951-b5f4-e6eff7611b15-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-1269b2-fdd4895d7-mr8fh\" (UID: \"a17e2253-de02-4951-b5f4-e6eff7611b15\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-1269b2-fdd4895d7-mr8fh"
Sep 17 14:18:57.395582 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:57.395557    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a17e2253-de02-4951-b5f4-e6eff7611b15-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-1269b2-fdd4895d7-mr8fh\" (UID: \"a17e2253-de02-4951-b5f4-e6eff7611b15\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-1269b2-fdd4895d7-mr8fh"
Sep 17 14:18:57.402706 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:57.402688    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8pc4\" (UniqueName: \"kubernetes.io/projected/a17e2253-de02-4951-b5f4-e6eff7611b15-kube-api-access-d8pc4\") pod \"payload-processing-e2e-worker-w0-1269b2-fdd4895d7-mr8fh\" (UID: \"a17e2253-de02-4951-b5f4-e6eff7611b15\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-1269b2-fdd4895d7-mr8fh"
Sep 17 14:18:57.598353 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:57.598323    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-1269b2-fdd4895d7-mr8fh"
Sep 17 14:18:57.708952 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:57.708848    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:18:57.711235 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:57.711198    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7877-e34dd9" podUID="65ccf32c-3fea-47ff-a18b-65fe74b0ca3c" containerName="test-curl-7877-e34dd9" containerID="cri-o://45929a9a93389f7661fb92d05cd6cf1c81b4abe30e34cb42e9d6b4f3f3f02dd3" gracePeriod=2
Sep 17 14:18:57.809034 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:57.808994    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:18:57.879738 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:57.879714    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-1269b2-fdd4895d7-mr8fh"]
Sep 17 14:18:57.885266 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:18:57.885237    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda17e2253_de02_4951_b5f4_e6eff7611b15.slice/crio-72d1cf01228dd39bf520fa45f970aec7e3f0e70e83d642e536a66b799c1511bf WatchSource:0}: Error finding container 72d1cf01228dd39bf520fa45f970aec7e3f0e70e83d642e536a66b799c1511bf: Status 404 returned error can't find the container with id 72d1cf01228dd39bf520fa45f970aec7e3f0e70e83d642e536a66b799c1511bf
Sep 17 14:18:58.159968 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:58.159884    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:18:58.285490 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:58.285454    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:18:58.611074 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:58.611039    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:18:58.719533 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:58.719496    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-1269b2-fdd4895d7-mr8fh" event={"ID":"a17e2253-de02-4951-b5f4-e6eff7611b15","Type":"ContainerStarted","Data":"efae5a15053b5d195b0b07441717f32c9cb471089b38575c9ce5b2856db67071"}
Sep 17 14:18:58.720029 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:58.719555    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-1269b2-fdd4895d7-mr8fh" event={"ID":"a17e2253-de02-4951-b5f4-e6eff7611b15","Type":"ContainerStarted","Data":"72d1cf01228dd39bf520fa45f970aec7e3f0e70e83d642e536a66b799c1511bf"}
Sep 17 14:18:58.720029 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:58.719626    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-1269b2-fdd4895d7-mr8fh"
Sep 17 14:18:58.729755 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:58.729724    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:18:58.738753 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:58.738707    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-1269b2-fdd4895d7-mr8fh" podStartSLOduration=1.738690157 podStartE2EDuration="1.738690157s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:18:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:18:58.737880471 +0000 UTC m=+839.739423947" watchObservedRunningTime="2026-09-17 14:18:58.738690157 +0000 UTC m=+839.740233631"
Sep 17 14:18:58.842377 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:58.842331    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:18:59.043859 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:59.043816    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:18:59.129423 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:59.129382    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-181b16-55c996cb6-prk6x"]
Sep 17 14:18:59.129875 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:59.129836    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:18:59.134036 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:59.134015    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-181b16-55c996cb6-prk6x"
Sep 17 14:18:59.138926 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:59.138904    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-181b16-dockercfg-px7xd\""
Sep 17 14:18:59.139061 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:59.138912    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-181b16\""
Sep 17 14:18:59.144810 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:59.144786    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-181b16-55c996cb6-prk6x"]
Sep 17 14:18:59.159789 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:59.159751    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-181b16-bd664686c-hq5fv"]
Sep 17 14:18:59.164002 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:59.163983    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-181b16-bd664686c-hq5fv"
Sep 17 14:18:59.177217 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:59.177194    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-181b16-bd664686c-hq5fv"]
Sep 17 14:18:59.209553 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:59.209527    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tngg4\" (UniqueName: \"kubernetes.io/projected/72dead9f-9eb2-4ac2-a8af-a85ff63ee44f-kube-api-access-tngg4\") pod \"payload-pre-processing-e2e-inf-a-181b16-55c996cb6-prk6x\" (UID: \"72dead9f-9eb2-4ac2-a8af-a85ff63ee44f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-181b16-55c996cb6-prk6x"
Sep 17 14:18:59.209694 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:59.209579    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e285562e-f8c3-45a2-b537-aae94a204392-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-181b16-bd664686c-hq5fv\" (UID: \"e285562e-f8c3-45a2-b537-aae94a204392\") " pod="openshift-ingress/payload-processing-e2e-inf-a-181b16-bd664686c-hq5fv"
Sep 17 14:18:59.209694 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:59.209677    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpqjg\" (UniqueName: \"kubernetes.io/projected/e285562e-f8c3-45a2-b537-aae94a204392-kube-api-access-lpqjg\") pod \"payload-processing-e2e-inf-a-181b16-bd664686c-hq5fv\" (UID: \"e285562e-f8c3-45a2-b537-aae94a204392\") " pod="openshift-ingress/payload-processing-e2e-inf-a-181b16-bd664686c-hq5fv"
Sep 17 14:18:59.209776 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:59.209697    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72dead9f-9eb2-4ac2-a8af-a85ff63ee44f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-181b16-55c996cb6-prk6x\" (UID: \"72dead9f-9eb2-4ac2-a8af-a85ff63ee44f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-181b16-55c996cb6-prk6x"
Sep 17 14:18:59.310244 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:59.310141    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpqjg\" (UniqueName: \"kubernetes.io/projected/e285562e-f8c3-45a2-b537-aae94a204392-kube-api-access-lpqjg\") pod \"payload-processing-e2e-inf-a-181b16-bd664686c-hq5fv\" (UID: \"e285562e-f8c3-45a2-b537-aae94a204392\") " pod="openshift-ingress/payload-processing-e2e-inf-a-181b16-bd664686c-hq5fv"
Sep 17 14:18:59.310244 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:59.310183    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72dead9f-9eb2-4ac2-a8af-a85ff63ee44f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-181b16-55c996cb6-prk6x\" (UID: \"72dead9f-9eb2-4ac2-a8af-a85ff63ee44f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-181b16-55c996cb6-prk6x"
Sep 17 14:18:59.310499 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:59.310287    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tngg4\" (UniqueName: \"kubernetes.io/projected/72dead9f-9eb2-4ac2-a8af-a85ff63ee44f-kube-api-access-tngg4\") pod \"payload-pre-processing-e2e-inf-a-181b16-55c996cb6-prk6x\" (UID: \"72dead9f-9eb2-4ac2-a8af-a85ff63ee44f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-181b16-55c996cb6-prk6x"
Sep 17 14:18:59.310499 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:59.310343    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e285562e-f8c3-45a2-b537-aae94a204392-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-181b16-bd664686c-hq5fv\" (UID: \"e285562e-f8c3-45a2-b537-aae94a204392\") " pod="openshift-ingress/payload-processing-e2e-inf-a-181b16-bd664686c-hq5fv"
Sep 17 14:18:59.310922 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:59.310896    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72dead9f-9eb2-4ac2-a8af-a85ff63ee44f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-181b16-55c996cb6-prk6x\" (UID: \"72dead9f-9eb2-4ac2-a8af-a85ff63ee44f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-181b16-55c996cb6-prk6x"
Sep 17 14:18:59.311101 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:59.311079    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e285562e-f8c3-45a2-b537-aae94a204392-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-181b16-bd664686c-hq5fv\" (UID: \"e285562e-f8c3-45a2-b537-aae94a204392\") " pod="openshift-ingress/payload-processing-e2e-inf-a-181b16-bd664686c-hq5fv"
Sep 17 14:18:59.318095 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:59.318073    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpqjg\" (UniqueName: \"kubernetes.io/projected/e285562e-f8c3-45a2-b537-aae94a204392-kube-api-access-lpqjg\") pod \"payload-processing-e2e-inf-a-181b16-bd664686c-hq5fv\" (UID: \"e285562e-f8c3-45a2-b537-aae94a204392\") " pod="openshift-ingress/payload-processing-e2e-inf-a-181b16-bd664686c-hq5fv"
Sep 17 14:18:59.318194 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:59.318171    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tngg4\" (UniqueName: \"kubernetes.io/projected/72dead9f-9eb2-4ac2-a8af-a85ff63ee44f-kube-api-access-tngg4\") pod \"payload-pre-processing-e2e-inf-a-181b16-55c996cb6-prk6x\" (UID: \"72dead9f-9eb2-4ac2-a8af-a85ff63ee44f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-181b16-55c996cb6-prk6x"
Sep 17 14:18:59.448937 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:59.448895    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-181b16-dockercfg-px7xd\""
Sep 17 14:18:59.456678 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:59.456660    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-181b16-55c996cb6-prk6x"
Sep 17 14:18:59.473795 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:59.473766    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-181b16-bd664686c-hq5fv"
Sep 17 14:18:59.614789 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:59.614689    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:18:59.739586 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:59.739560    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-181b16-55c996cb6-prk6x"]
Sep 17 14:18:59.764884 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:18:59.764841    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72dead9f_9eb2_4ac2_a8af_a85ff63ee44f.slice/crio-3e7605605a56ac3276f7faf8d0df9d6e63ecc4e85b6a1ffada5fb846cff50c1d WatchSource:0}: Error finding container 3e7605605a56ac3276f7faf8d0df9d6e63ecc4e85b6a1ffada5fb846cff50c1d: Status 404 returned error can't find the container with id 3e7605605a56ac3276f7faf8d0df9d6e63ecc4e85b6a1ffada5fb846cff50c1d
Sep 17 14:18:59.764998 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:59.764966    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-181b16-bd664686c-hq5fv"]
Sep 17 14:18:59.768135 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:18:59.768112    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode285562e_f8c3_45a2_b537_aae94a204392.slice/crio-c4495959ed79c434bfd80bed640fd9d050cee7242cbb995dc418c8dc4fe79cfb WatchSource:0}: Error finding container c4495959ed79c434bfd80bed640fd9d050cee7242cbb995dc418c8dc4fe79cfb: Status 404 returned error can't find the container with id c4495959ed79c434bfd80bed640fd9d050cee7242cbb995dc418c8dc4fe79cfb
Sep 17 14:18:59.909010 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:59.908984    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7877-e34dd9"
Sep 17 14:18:59.914875 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:59.914845    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z6fvx\" (UniqueName: \"kubernetes.io/projected/65ccf32c-3fea-47ff-a18b-65fe74b0ca3c-kube-api-access-z6fvx\") pod \"65ccf32c-3fea-47ff-a18b-65fe74b0ca3c\" (UID: \"65ccf32c-3fea-47ff-a18b-65fe74b0ca3c\") "
Sep 17 14:18:59.917990 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:18:59.917878    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65ccf32c-3fea-47ff-a18b-65fe74b0ca3c-kube-api-access-z6fvx" (OuterVolumeSpecName: "kube-api-access-z6fvx") pod "65ccf32c-3fea-47ff-a18b-65fe74b0ca3c" (UID: "65ccf32c-3fea-47ff-a18b-65fe74b0ca3c"). InnerVolumeSpecName "kube-api-access-z6fvx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:19:00.015760 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:00.015721    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z6fvx\" (UniqueName: \"kubernetes.io/projected/65ccf32c-3fea-47ff-a18b-65fe74b0ca3c-kube-api-access-z6fvx\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:19:00.360780 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:00.360749    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:00.468781 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:00.468740    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:00.584208 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:00.584115    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:00.734550 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:00.734499    2582 generic.go:356] "Generic (PLEG): container finished" podID="65ccf32c-3fea-47ff-a18b-65fe74b0ca3c" containerID="45929a9a93389f7661fb92d05cd6cf1c81b4abe30e34cb42e9d6b4f3f3f02dd3" exitCode=137
Sep 17 14:19:00.734859 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:00.734600    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7877-e34dd9"
Sep 17 14:19:00.734977 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:00.734649    2582 scope.go:117] "RemoveContainer" containerID="45929a9a93389f7661fb92d05cd6cf1c81b4abe30e34cb42e9d6b4f3f3f02dd3"
Sep 17 14:19:00.737338 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:00.737309    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-181b16-bd664686c-hq5fv" event={"ID":"e285562e-f8c3-45a2-b537-aae94a204392","Type":"ContainerStarted","Data":"ab17cb81e5eabc547b8971ff3eb6f32bb85b2f2539d5bdc356d0f13792d3988d"}
Sep 17 14:19:00.737443 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:00.737366    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-181b16-bd664686c-hq5fv" event={"ID":"e285562e-f8c3-45a2-b537-aae94a204392","Type":"ContainerStarted","Data":"c4495959ed79c434bfd80bed640fd9d050cee7242cbb995dc418c8dc4fe79cfb"}
Sep 17 14:19:00.738243 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:00.738081    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-181b16-bd664686c-hq5fv"
Sep 17 14:19:00.740471 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:00.740441    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-181b16-55c996cb6-prk6x" event={"ID":"72dead9f-9eb2-4ac2-a8af-a85ff63ee44f","Type":"ContainerStarted","Data":"0d4759687bb68a5ac51648f6a3a4a09e173cb98300f2973a7d887f6cbaf734cc"}
Sep 17 14:19:00.740861 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:00.740492    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-181b16-55c996cb6-prk6x" event={"ID":"72dead9f-9eb2-4ac2-a8af-a85ff63ee44f","Type":"ContainerStarted","Data":"3e7605605a56ac3276f7faf8d0df9d6e63ecc4e85b6a1ffada5fb846cff50c1d"}
Sep 17 14:19:00.740861 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:00.740686    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-181b16-55c996cb6-prk6x"
Sep 17 14:19:00.747277 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:00.747255    2582 scope.go:117] "RemoveContainer" containerID="45929a9a93389f7661fb92d05cd6cf1c81b4abe30e34cb42e9d6b4f3f3f02dd3"
Sep 17 14:19:00.747576 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:19:00.747549    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45929a9a93389f7661fb92d05cd6cf1c81b4abe30e34cb42e9d6b4f3f3f02dd3\": container with ID starting with 45929a9a93389f7661fb92d05cd6cf1c81b4abe30e34cb42e9d6b4f3f3f02dd3 not found: ID does not exist" containerID="45929a9a93389f7661fb92d05cd6cf1c81b4abe30e34cb42e9d6b4f3f3f02dd3"
Sep 17 14:19:00.747669 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:00.747584    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45929a9a93389f7661fb92d05cd6cf1c81b4abe30e34cb42e9d6b4f3f3f02dd3"} err="failed to get container status \"45929a9a93389f7661fb92d05cd6cf1c81b4abe30e34cb42e9d6b4f3f3f02dd3\": rpc error: code = NotFound desc = could not find container \"45929a9a93389f7661fb92d05cd6cf1c81b4abe30e34cb42e9d6b4f3f3f02dd3\": container with ID starting with 45929a9a93389f7661fb92d05cd6cf1c81b4abe30e34cb42e9d6b4f3f3f02dd3 not found: ID does not exist"
Sep 17 14:19:00.757113 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:19:00.757084    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7877-e34dd9\" is forbidden: User \"system:node:ip-10-0-137-53.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-53.ec2.internal' and this object" podUID="65ccf32c-3fea-47ff-a18b-65fe74b0ca3c" pod="opendatahub/test-curl-7877-e34dd9"
Sep 17 14:19:00.758955 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:00.758898    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-181b16-bd664686c-hq5fv" podStartSLOduration=1.7588814720000001 podStartE2EDuration="1.758881472s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:18:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:19:00.755388996 +0000 UTC m=+841.756932475" watchObservedRunningTime="2026-09-17 14:19:00.758881472 +0000 UTC m=+841.760424950"
Sep 17 14:19:00.784636 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:00.784477    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-181b16-55c996cb6-prk6x" podStartSLOduration=1.784461702 podStartE2EDuration="1.784461702s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:18:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:19:00.782779162 +0000 UTC m=+841.784322638" watchObservedRunningTime="2026-09-17 14:19:00.784461702 +0000 UTC m=+841.786005175"
Sep 17 14:19:00.873023 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:00.872940    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:01.063463 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:01.063432    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:01.337459 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:01.337426    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:01.495107 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:01.495073    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65ccf32c-3fea-47ff-a18b-65fe74b0ca3c" path="/var/lib/kubelet/pods/65ccf32c-3fea-47ff-a18b-65fe74b0ca3c/volumes"
Sep 17 14:19:02.687802 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:02.687762    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:02.967767 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:02.967731    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm"]
Sep 17 14:19:02.972890 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:02.972865    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm"
Sep 17 14:19:02.975841 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:02.975817    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-3f6f32-openshift-default-dockercfg-qvmzw\""
Sep 17 14:19:02.985348 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:02.985326    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm"]
Sep 17 14:19:02.999299 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:02.999273    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:03.051181 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:03.051139    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-istiod-ca-cert\") pod \"e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm\" (UID: \"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c\") " pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm"
Sep 17 14:19:03.051181 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:03.051183    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-workload-socket\") pod \"e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm\" (UID: \"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c\") " pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm"
Sep 17 14:19:03.051417 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:03.051226    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-credential-socket\") pod \"e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm\" (UID: \"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c\") " pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm"
Sep 17 14:19:03.051417 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:03.051269    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-istio-token\") pod \"e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm\" (UID: \"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c\") " pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm"
Sep 17 14:19:03.051417 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:03.051302    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-istio-envoy\") pod \"e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm\" (UID: \"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c\") " pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm"
Sep 17 14:19:03.051417 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:03.051327    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-workload-certs\") pod \"e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm\" (UID: \"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c\") " pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm"
Sep 17 14:19:03.051417 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:03.051359    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-istio-podinfo\") pod \"e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm\" (UID: \"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c\") " pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm"
Sep 17 14:19:03.051417 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:03.051405    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkxbk\" (UniqueName: \"kubernetes.io/projected/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-kube-api-access-wkxbk\") pod \"e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm\" (UID: \"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c\") " pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm"
Sep 17 14:19:03.051651 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:03.051431    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-istio-data\") pod \"e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm\" (UID: \"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c\") " pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm"
Sep 17 14:19:03.152946 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:03.152904    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-istio-podinfo\") pod \"e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm\" (UID: \"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c\") " pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm"
Sep 17 14:19:03.152946 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:03.152946    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkxbk\" (UniqueName: \"kubernetes.io/projected/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-kube-api-access-wkxbk\") pod \"e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm\" (UID: \"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c\") " pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm"
Sep 17 14:19:03.153226 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:03.152968    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-istio-data\") pod \"e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm\" (UID: \"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c\") " pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm"
Sep 17 14:19:03.153226 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:03.153005    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-istiod-ca-cert\") pod \"e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm\" (UID: \"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c\") " pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm"
Sep 17 14:19:03.153226 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:03.153026    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-workload-socket\") pod \"e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm\" (UID: \"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c\") " pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm"
Sep 17 14:19:03.153226 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:03.153062    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-credential-socket\") pod \"e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm\" (UID: \"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c\") " pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm"
Sep 17 14:19:03.153226 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:03.153121    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-istio-token\") pod \"e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm\" (UID: \"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c\") " pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm"
Sep 17 14:19:03.153226 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:03.153158    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-istio-envoy\") pod \"e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm\" (UID: \"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c\") " pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm"
Sep 17 14:19:03.153226 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:03.153190    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-workload-certs\") pod \"e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm\" (UID: \"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c\") " pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm"
Sep 17 14:19:03.153673 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:03.153538    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-istio-data\") pod \"e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm\" (UID: \"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c\") " pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm"
Sep 17 14:19:03.153750 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:03.153726    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-workload-certs\") pod \"e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm\" (UID: \"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c\") " pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm"
Sep 17 14:19:03.153839 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:03.153818    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-workload-socket\") pod \"e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm\" (UID: \"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c\") " pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm"
Sep 17 14:19:03.153936 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:03.153914    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-credential-socket\") pod \"e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm\" (UID: \"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c\") " pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm"
Sep 17 14:19:03.153984 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:03.153954    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-istiod-ca-cert\") pod \"e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm\" (UID: \"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c\") " pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm"
Sep 17 14:19:03.155565 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:03.155548    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-istio-podinfo\") pod \"e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm\" (UID: \"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c\") " pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm"
Sep 17 14:19:03.155693 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:03.155677    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-istio-envoy\") pod \"e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm\" (UID: \"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c\") " pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm"
Sep 17 14:19:03.161331 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:03.161304    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkxbk\" (UniqueName: \"kubernetes.io/projected/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-kube-api-access-wkxbk\") pod \"e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm\" (UID: \"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c\") " pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm"
Sep 17 14:19:03.161451 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:03.161419    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-istio-token\") pod \"e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm\" (UID: \"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c\") " pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm"
Sep 17 14:19:03.284872 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:03.284783    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm"
Sep 17 14:19:03.296073 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:03.296039    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:03.573247 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:03.573152    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm"]
Sep 17 14:19:03.578205 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:19:03.578158    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f56f2e1_a1f7_4512_8024_7ab51ac41a9c.slice/crio-c18892ff6e8b79409ff8b49068bceaad60248be9d8c467ff045ca9efe5095b69 WatchSource:0}: Error finding container c18892ff6e8b79409ff8b49068bceaad60248be9d8c467ff045ca9efe5095b69: Status 404 returned error can't find the container with id c18892ff6e8b79409ff8b49068bceaad60248be9d8c467ff045ca9efe5095b69
Sep 17 14:19:03.580889 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:03.580853    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:19:03.581020 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:03.580944    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:19:03.581020 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:03.580989    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:19:03.762936 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:03.762901    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm" event={"ID":"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c","Type":"ContainerStarted","Data":"cbd65e276f0944ef708a40b87442b53da7c76bebb218ac24d0c9bc1306177057"}
Sep 17 14:19:03.763324 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:03.762961    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm" event={"ID":"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c","Type":"ContainerStarted","Data":"c18892ff6e8b79409ff8b49068bceaad60248be9d8c467ff045ca9efe5095b69"}
Sep 17 14:19:03.786580 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:03.786514    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm" podStartSLOduration=1.786490656 podStartE2EDuration="1.786490656s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:19:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:19:03.782950018 +0000 UTC m=+844.784493492" watchObservedRunningTime="2026-09-17 14:19:03.786490656 +0000 UTC m=+844.788034132"
Sep 17 14:19:04.285709 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:04.285668    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm"
Sep 17 14:19:04.287696 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:04.287666    2582 patch_prober.go:28] interesting pod/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 17 14:19:04.287804 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:04.287723    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm" podUID="7f56f2e1-a1f7-4512-8024-7ab51ac41a9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 17 14:19:05.027670 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:05.027601    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:05.290019 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:05.289943    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm"
Sep 17 14:19:05.719397 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:05.719362    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:05.777740 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:05.777703    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm"
Sep 17 14:19:05.778836 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:05.778812    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm"
Sep 17 14:19:05.917464 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:05.917429    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:06.005363 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:06.005264    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:06.112187 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:06.112144    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:06.205213 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:06.205174    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:06.876632 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:06.867557    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:07.767181 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:07.767135    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:08.537403 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:08.537284    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:08.606759 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:08.606727    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:08.820860 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:08.820745    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:09.193686 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:09.187714    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:09.307634 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:09.306743    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:09.487685 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:09.485178    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:09.728170 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:09.728127    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-1269b2-fdd4895d7-mr8fh"
Sep 17 14:19:09.731187 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:09.731158    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:09.889692 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:09.884870    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:09.988147 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:09.988114    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:10.084964 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:10.084911    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:10.565168 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:10.565126    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:10.651055 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:10.651016    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:10.752011 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:10.751978    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:11.122651 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:11.121620    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:11.225505 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:11.225463    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:11.230647 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:11.227309    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-3f6f32-796794d758-5sscr"]
Sep 17 14:19:11.252907 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:11.252873    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-3f6f32-796794d758-5sscr"]
Sep 17 14:19:11.253211 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:11.253193    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-3f6f32-796794d758-5sscr"
Sep 17 14:19:11.260261 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:11.260228    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-3f6f32-dockercfg-9r7dw\""
Sep 17 14:19:11.260567 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:11.260544    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-3f6f32\""
Sep 17 14:19:11.306632 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:11.306322    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txwgq\" (UniqueName: \"kubernetes.io/projected/e09c655f-30c2-4939-a73b-03ecdf25ff5a-kube-api-access-txwgq\") pod \"payload-pre-processing-e2e-autores-3f6f32-796794d758-5sscr\" (UID: \"e09c655f-30c2-4939-a73b-03ecdf25ff5a\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-3f6f32-796794d758-5sscr"
Sep 17 14:19:11.306632 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:11.306406    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e09c655f-30c2-4939-a73b-03ecdf25ff5a-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-3f6f32-796794d758-5sscr\" (UID: \"e09c655f-30c2-4939-a73b-03ecdf25ff5a\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-3f6f32-796794d758-5sscr"
Sep 17 14:19:11.409339 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:11.409228    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e09c655f-30c2-4939-a73b-03ecdf25ff5a-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-3f6f32-796794d758-5sscr\" (UID: \"e09c655f-30c2-4939-a73b-03ecdf25ff5a\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-3f6f32-796794d758-5sscr"
Sep 17 14:19:11.409772 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:11.409745    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txwgq\" (UniqueName: \"kubernetes.io/projected/e09c655f-30c2-4939-a73b-03ecdf25ff5a-kube-api-access-txwgq\") pod \"payload-pre-processing-e2e-autores-3f6f32-796794d758-5sscr\" (UID: \"e09c655f-30c2-4939-a73b-03ecdf25ff5a\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-3f6f32-796794d758-5sscr"
Sep 17 14:19:11.411246 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:11.411220    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e09c655f-30c2-4939-a73b-03ecdf25ff5a-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-3f6f32-796794d758-5sscr\" (UID: \"e09c655f-30c2-4939-a73b-03ecdf25ff5a\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-3f6f32-796794d758-5sscr"
Sep 17 14:19:11.421488 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:11.419685    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txwgq\" (UniqueName: \"kubernetes.io/projected/e09c655f-30c2-4939-a73b-03ecdf25ff5a-kube-api-access-txwgq\") pod \"payload-pre-processing-e2e-autores-3f6f32-796794d758-5sscr\" (UID: \"e09c655f-30c2-4939-a73b-03ecdf25ff5a\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-3f6f32-796794d758-5sscr"
Sep 17 14:19:11.584367 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:11.584331    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:11.616460 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:11.616430    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-3f6f32-796794d758-5sscr"
Sep 17 14:19:11.760635 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:11.754753    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-181b16-55c996cb6-prk6x"
Sep 17 14:19:11.933264 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:11.933229    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-3f6f32-796794d758-5sscr"]
Sep 17 14:19:11.964294 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:19:11.964208    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode09c655f_30c2_4939_a73b_03ecdf25ff5a.slice/crio-85b09fa7ff4a65a6cff832fd72553931516937d0ae3cedb89f9092faccef3685 WatchSource:0}: Error finding container 85b09fa7ff4a65a6cff832fd72553931516937d0ae3cedb89f9092faccef3685: Status 404 returned error can't find the container with id 85b09fa7ff4a65a6cff832fd72553931516937d0ae3cedb89f9092faccef3685
Sep 17 14:19:12.457298 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:12.457261    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:12.680787 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:12.680706    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:12.758831 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:12.758714    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-181b16-bd664686c-hq5fv"
Sep 17 14:19:12.845011 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:12.836070    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:12.879537 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:12.879498    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-3f6f32-796794d758-5sscr" event={"ID":"e09c655f-30c2-4939-a73b-03ecdf25ff5a","Type":"ContainerStarted","Data":"036b8edc50a8ab8611ffbc03ae61fb2b9a0b5227113a5c9216c1d5771e14a99b"}
Sep 17 14:19:12.880150 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:12.880128    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-3f6f32-796794d758-5sscr"
Sep 17 14:19:12.880350 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:12.880332    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-3f6f32-796794d758-5sscr" event={"ID":"e09c655f-30c2-4939-a73b-03ecdf25ff5a","Type":"ContainerStarted","Data":"85b09fa7ff4a65a6cff832fd72553931516937d0ae3cedb89f9092faccef3685"}
Sep 17 14:19:12.898843 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:12.898374    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-3f6f32-796794d758-5sscr" podStartSLOduration=1.898332719 podStartE2EDuration="1.898332719s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:19:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:19:12.896889512 +0000 UTC m=+853.898433022" watchObservedRunningTime="2026-09-17 14:19:12.898332719 +0000 UTC m=+853.899876193"
Sep 17 14:19:12.908955 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:12.908916    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:13.000567 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:13.000527    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:13.242534 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:13.242495    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:14.208640 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:14.208547    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:14.573949 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:14.573862    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:14.674021 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:14.673982    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:15.009917 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:15.009881    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:15.183221 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:15.183178    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:15.384841 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:15.384123    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:15.756570 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:15.756531    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:16.002823 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:16.002783    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:16.103191 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:16.103112    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:16.760040 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:16.759996    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:16.907552 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:16.907515    2582 generic.go:356] "Generic (PLEG): container finished" podID="725b0118-b60a-4972-a84a-d52af1642e0b" containerID="a7dfc2b887f2083f84aa4641efa503bcf0a4325a9468bdfd34fd53fda29ef71b" exitCode=0
Sep 17 14:19:16.907726 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:16.907602    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn" event={"ID":"725b0118-b60a-4972-a84a-d52af1642e0b","Type":"ContainerDied","Data":"a7dfc2b887f2083f84aa4641efa503bcf0a4325a9468bdfd34fd53fda29ef71b"}
Sep 17 14:19:16.908200 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:16.908170    2582 scope.go:117] "RemoveContainer" containerID="a7dfc2b887f2083f84aa4641efa503bcf0a4325a9468bdfd34fd53fda29ef71b"
Sep 17 14:19:16.909323 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:16.909303    2582 generic.go:356] "Generic (PLEG): container finished" podID="368ca806-07df-48be-8ee6-3e8773114611" containerID="7000bfc51c007aaa54be164e8b608517d22220b1155dd9ec6af3c4fe1501b0c9" exitCode=0
Sep 17 14:19:16.909417 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:16.909368    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6" event={"ID":"368ca806-07df-48be-8ee6-3e8773114611","Type":"ContainerDied","Data":"7000bfc51c007aaa54be164e8b608517d22220b1155dd9ec6af3c4fe1501b0c9"}
Sep 17 14:19:16.909417 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:16.909374    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:19:16.909529 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:16.909439    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:19:16.909529 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:16.909469    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:19:16.910407 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:16.910305    2582 scope.go:117] "RemoveContainer" containerID="7000bfc51c007aaa54be164e8b608517d22220b1155dd9ec6af3c4fe1501b0c9"
Sep 17 14:19:16.911588 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:16.911556    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:19:16.911723 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:16.911654    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:19:16.911723 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:16.911705    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:19:17.743899 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:17.743860    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:17.843463 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:17.843385    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:17.919551 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:17.919501    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6" event={"ID":"368ca806-07df-48be-8ee6-3e8773114611","Type":"ContainerStarted","Data":"b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09"}
Sep 17 14:19:17.922064 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:17.922038    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn" event={"ID":"725b0118-b60a-4972-a84a-d52af1642e0b","Type":"ContainerStarted","Data":"0af183ee7dd9bde16635b6bc482cde3e139067e5bc2acaa05ddfaf1e2a2afd24"}
Sep 17 14:19:18.086070 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:18.085953    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:19:18.086343 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:18.086311    2582 patch_prober.go:28] interesting pod/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 17 14:19:18.086447 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:18.086399    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6" podUID="368ca806-07df-48be-8ee6-3e8773114611" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 17 14:19:18.165191 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:18.165140    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn"
Sep 17 14:19:18.165505 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:18.165469    2582 patch_prober.go:28] interesting pod/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Sep 17 14:19:18.165644 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:18.165559    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn" podUID="725b0118-b60a-4972-a84a-d52af1642e0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Sep 17 14:19:18.633859 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:18.633816    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:18.932278 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:18.929632    2582 generic.go:356] "Generic (PLEG): container finished" podID="68dd2b1c-ed84-4291-8fb7-f61779fd5b85" containerID="ed106bce24d830f2308cc17efa0004d937f823952dc96ce55a4beb200a5caf86" exitCode=0
Sep 17 14:19:18.932278 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:18.930643    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md" event={"ID":"68dd2b1c-ed84-4291-8fb7-f61779fd5b85","Type":"ContainerDied","Data":"ed106bce24d830f2308cc17efa0004d937f823952dc96ce55a4beb200a5caf86"}
Sep 17 14:19:18.932278 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:18.931056    2582 scope.go:117] "RemoveContainer" containerID="ed106bce24d830f2308cc17efa0004d937f823952dc96ce55a4beb200a5caf86"
Sep 17 14:19:18.934750 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:18.934390    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:19:18.934750 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:18.934483    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:19:18.934750 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:18.934534    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:19:19.029630 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:19.023447    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:19.087919 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:19.087406    2582 patch_prober.go:28] interesting pod/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 17 14:19:19.087919 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:19.087471    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6" podUID="368ca806-07df-48be-8ee6-3e8773114611" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 17 14:19:19.126786 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:19.126666    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:19.173633 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:19.165014    2582 patch_prober.go:28] interesting pod/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Sep 17 14:19:19.173633 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:19.165078    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn" podUID="725b0118-b60a-4972-a84a-d52af1642e0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Sep 17 14:19:19.434298 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:19.434254    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:19.941951 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:19.941904    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md" event={"ID":"68dd2b1c-ed84-4291-8fb7-f61779fd5b85","Type":"ContainerStarted","Data":"e1a29482fea6e73c670a158f9ff4259d158ff57276aa747806fb8ea0d330d790"}
Sep 17 14:19:20.019522 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:20.019481    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md"
Sep 17 14:19:20.019738 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:20.019705    2582 patch_prober.go:28] interesting pod/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 17 14:19:20.019820 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:20.019770    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md" podUID="68dd2b1c-ed84-4291-8fb7-f61779fd5b85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 17 14:19:20.086287 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:20.086244    2582 patch_prober.go:28] interesting pod/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 17 14:19:20.086479 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:20.086317    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6" podUID="368ca806-07df-48be-8ee6-3e8773114611" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 17 14:19:20.165395 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:20.165347    2582 patch_prober.go:28] interesting pod/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Sep 17 14:19:20.165586 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:20.165416    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn" podUID="725b0118-b60a-4972-a84a-d52af1642e0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Sep 17 14:19:20.245882 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:20.245740    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:20.343013 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:20.342975    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:20.441140 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:20.441099    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:20.618062 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:20.617964    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:20.732439 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:20.732396    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:21.018246 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:21.018197    2582 patch_prober.go:28] interesting pod/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 17 14:19:21.018890 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:21.018841    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md" podUID="68dd2b1c-ed84-4291-8fb7-f61779fd5b85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 17 14:19:21.086220 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:21.086166    2582 patch_prober.go:28] interesting pod/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 17 14:19:21.086541 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:21.086491    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6" podUID="368ca806-07df-48be-8ee6-3e8773114611" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 17 14:19:21.169633 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:21.168086    2582 patch_prober.go:28] interesting pod/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Sep 17 14:19:21.169633 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:21.168154    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn" podUID="725b0118-b60a-4972-a84a-d52af1642e0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Sep 17 14:19:21.304718 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:21.303122    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:21.405631 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:21.404137    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:21.592115 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:21.592019    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ccd499b1-znct8"]
Sep 17 14:19:21.599517 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:21.599473    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ccd499b1-znct8"
Sep 17 14:19:21.602536 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:21.602502    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ddwrn\""
Sep 17 14:19:21.606997 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:21.606964    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ccd499b1-znct8"]
Sep 17 14:19:21.669662 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:21.669569    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/853a644f-4184-47ef-863e-c08ac7dbee38-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ccd499b1-znct8\" (UID: \"853a644f-4184-47ef-863e-c08ac7dbee38\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ccd499b1-znct8"
Sep 17 14:19:21.773077 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:21.772195    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/853a644f-4184-47ef-863e-c08ac7dbee38-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ccd499b1-znct8\" (UID: \"853a644f-4184-47ef-863e-c08ac7dbee38\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ccd499b1-znct8"
Sep 17 14:19:21.773077 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:21.773022    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/853a644f-4184-47ef-863e-c08ac7dbee38-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ccd499b1-znct8\" (UID: \"853a644f-4184-47ef-863e-c08ac7dbee38\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ccd499b1-znct8"
Sep 17 14:19:21.870791 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:21.866502    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:21.917277 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:21.917040    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ccd499b1-znct8"
Sep 17 14:19:22.028632 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:22.026314    2582 patch_prober.go:28] interesting pod/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 17 14:19:22.028632 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:22.026409    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md" podUID="68dd2b1c-ed84-4291-8fb7-f61779fd5b85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 17 14:19:22.090633 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:22.088987    2582 patch_prober.go:28] interesting pod/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 17 14:19:22.090633 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:22.089072    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6" podUID="368ca806-07df-48be-8ee6-3e8773114611" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 17 14:19:22.165340 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:22.165232    2582 patch_prober.go:28] interesting pod/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Sep 17 14:19:22.165340 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:22.165311    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn" podUID="725b0118-b60a-4972-a84a-d52af1642e0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Sep 17 14:19:22.447019 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:22.443847    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ccd499b1-znct8"]
Sep 17 14:19:22.480628 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:22.478296    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:23.003064 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:23.003011    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ccd499b1-znct8" event={"ID":"853a644f-4184-47ef-863e-c08ac7dbee38","Type":"ContainerStarted","Data":"ebf386f354bf687d181de0573c04cd1279741f5381abbe2a608136f7e728628b"}
Sep 17 14:19:23.018041 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:23.018003    2582 patch_prober.go:28] interesting pod/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 17 14:19:23.018213 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:23.018059    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md" podUID="68dd2b1c-ed84-4291-8fb7-f61779fd5b85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 17 14:19:23.087038 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:23.086550    2582 patch_prober.go:28] interesting pod/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 17 14:19:23.087038 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:23.086651    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6" podUID="368ca806-07df-48be-8ee6-3e8773114611" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 17 14:19:23.165006 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:23.164943    2582 patch_prober.go:28] interesting pod/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Sep 17 14:19:23.165299 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:23.165249    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn" podUID="725b0118-b60a-4972-a84a-d52af1642e0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Sep 17 14:19:23.166539 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:23.166420    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:23.685663 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:23.684739    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md"
Sep 17 14:19:23.686661 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:23.685921    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn"
Sep 17 14:19:23.686782 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:23.686666    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:19:24.018268 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:24.018179    2582 patch_prober.go:28] interesting pod/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 17 14:19:24.018268 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:24.018261    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md" podUID="68dd2b1c-ed84-4291-8fb7-f61779fd5b85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 17 14:19:24.035744 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:24.035664    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:24.245648 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:24.245201    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:19:24.255630 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:24.250233    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:19:24.350682 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:24.348482    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn"
Sep 17 14:19:24.357650 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:24.353852    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn"
Sep 17 14:19:24.875248 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:24.875215    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:24.897890 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:24.897792    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-3f6f32-796794d758-5sscr"
Sep 17 14:19:25.018638 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:25.018303    2582 patch_prober.go:28] interesting pod/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 17 14:19:25.018638 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:25.018352    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md" podUID="68dd2b1c-ed84-4291-8fb7-f61779fd5b85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 17 14:19:25.520696 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:25.520655    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:25.618928 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:25.618894    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:26.018280 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:26.018249    2582 patch_prober.go:28] interesting pod/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 17 14:19:26.018419 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:26.018301    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md" podUID="68dd2b1c-ed84-4291-8fb7-f61779fd5b85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 17 14:19:26.538550 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:26.538511    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:27.022732 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:27.022693    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md"
Sep 17 14:19:27.023859 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:27.023823    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md"
Sep 17 14:19:27.037889 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:27.037852    2582 generic.go:356] "Generic (PLEG): container finished" podID="853a644f-4184-47ef-863e-c08ac7dbee38" containerID="5fcb719a644bd0018bcc09a950d608bf1727bf85439e709b466595b53d4c43e3" exitCode=0
Sep 17 14:19:27.038051 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:27.037937    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ccd499b1-znct8" event={"ID":"853a644f-4184-47ef-863e-c08ac7dbee38","Type":"ContainerDied","Data":"5fcb719a644bd0018bcc09a950d608bf1727bf85439e709b466595b53d4c43e3"}
Sep 17 14:19:27.228468 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:27.228431    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:27.581105 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:27.581003    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:27.681348 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:27.681309    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:27.780754 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:27.780721    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:27.917851 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:27.917762    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:28.045745 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:28.045714    2582 generic.go:356] "Generic (PLEG): container finished" podID="7f56f2e1-a1f7-4512-8024-7ab51ac41a9c" containerID="cbd65e276f0944ef708a40b87442b53da7c76bebb218ac24d0c9bc1306177057" exitCode=0
Sep 17 14:19:28.045910 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:28.045788    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm" event={"ID":"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c","Type":"ContainerDied","Data":"cbd65e276f0944ef708a40b87442b53da7c76bebb218ac24d0c9bc1306177057"}
Sep 17 14:19:28.046576 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:28.046302    2582 scope.go:117] "RemoveContainer" containerID="cbd65e276f0944ef708a40b87442b53da7c76bebb218ac24d0c9bc1306177057"
Sep 17 14:19:28.047554 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:28.047523    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:19:28.047685 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:28.047595    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:19:28.047685 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:28.047651    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:19:28.201512 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:28.201489    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ccd499b1-znct8"
Sep 17 14:19:28.242320 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:28.242289    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:28.276898 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:28.276868    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/853a644f-4184-47ef-863e-c08ac7dbee38-maas-api-service-ca\") pod \"853a644f-4184-47ef-863e-c08ac7dbee38\" (UID: \"853a644f-4184-47ef-863e-c08ac7dbee38\") "
Sep 17 14:19:28.277298 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:28.277268    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/853a644f-4184-47ef-863e-c08ac7dbee38-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "853a644f-4184-47ef-863e-c08ac7dbee38" (UID: "853a644f-4184-47ef-863e-c08ac7dbee38"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:19:28.377930 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:28.377886    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/853a644f-4184-47ef-863e-c08ac7dbee38-maas-api-service-ca\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:19:28.945009 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:28.944978    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:29.053599 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:29.053560    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm" event={"ID":"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c","Type":"ContainerStarted","Data":"e4d988bd77a3919dd99840bf7ebe7d1685a5b85f8732061369386a8a1f7a8e94"}
Sep 17 14:19:29.054897 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:29.054874    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ccd499b1-znct8"
Sep 17 14:19:29.055017 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:29.054873    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ccd499b1-znct8" event={"ID":"853a644f-4184-47ef-863e-c08ac7dbee38","Type":"ContainerDied","Data":"ebf386f354bf687d181de0573c04cd1279741f5381abbe2a608136f7e728628b"}
Sep 17 14:19:29.055017 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:29.055001    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ebf386f354bf687d181de0573c04cd1279741f5381abbe2a608136f7e728628b"
Sep 17 14:19:29.285752 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:29.285671    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm"
Sep 17 14:19:29.285902 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:29.285881    2582 patch_prober.go:28] interesting pod/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 17 14:19:29.285946 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:29.285926    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm" podUID="7f56f2e1-a1f7-4512-8024-7ab51ac41a9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 17 14:19:29.634372 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:29.634249    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:30.285238 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:30.285203    2582 patch_prober.go:28] interesting pod/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 17 14:19:30.285661 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:30.285257    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm" podUID="7f56f2e1-a1f7-4512-8024-7ab51ac41a9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 17 14:19:30.430978 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:30.430939    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:31.285447 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:31.285406    2582 patch_prober.go:28] interesting pod/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 17 14:19:31.285932 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:31.285474    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm" podUID="7f56f2e1-a1f7-4512-8024-7ab51ac41a9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 17 14:19:31.534843 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:31.534790    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:32.084186 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:32.084134    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:32.170423 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:32.170387    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:32.253783 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:32.253745    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:32.285540 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:32.285485    2582 patch_prober.go:28] interesting pod/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 17 14:19:32.285951 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:32.285542    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm" podUID="7f56f2e1-a1f7-4512-8024-7ab51ac41a9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 17 14:19:32.286258 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:32.286233    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ccd499b1-znct8"]
Sep 17 14:19:32.288718 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:32.288696    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ccd499b1-znct8"]
Sep 17 14:19:32.322926 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:32.322893    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:32.444587 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:32.444547    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:32.593867 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:32.593807    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:32.706213 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:32.706103    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:33.001295 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:33.001205    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:33.099974 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:33.099939    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:33.201783 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:33.201749    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:33.286158 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:33.286065    2582 patch_prober.go:28] interesting pod/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 17 14:19:33.286158 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:33.286142    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm" podUID="7f56f2e1-a1f7-4512-8024-7ab51ac41a9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 17 14:19:33.471278 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:33.471236    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:33.495370 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:33.495335    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="853a644f-4184-47ef-863e-c08ac7dbee38" path="/var/lib/kubelet/pods/853a644f-4184-47ef-863e-c08ac7dbee38/volumes"
Sep 17 14:19:33.747541 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:33.747505    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:33.848788 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:33.848748    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:34.285116 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:34.285084    2582 patch_prober.go:28] interesting pod/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 17 14:19:34.285292 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:34.285136    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm" podUID="7f56f2e1-a1f7-4512-8024-7ab51ac41a9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 17 14:19:34.653752 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:34.653635    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:34.814263 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:34.814204    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:34.915281 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:34.915180    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:35.312949 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:35.312915    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm"
Sep 17 14:19:35.313339 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:35.313317    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm"
Sep 17 14:19:35.314478 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:35.314452    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm"
Sep 17 14:19:35.431106 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:35.431069    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:36.232550 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:36.232510    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:36.853330 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:36.853290    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:36.953873 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:36.953766    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:37.107590 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:37.107555    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:38.177570 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:38.177537    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:38.828278 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:38.828239    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:38.927844 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:38.927801    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:39.543137 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:39.543097    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:39.619383 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:39.619349    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:39.718145 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:39.718111    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:39.996075 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:39.996044    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:40.088814 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:40.088780    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:40.530539 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:40.530503    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:41.259146 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:41.259110    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:41.930782 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:41.930750    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:42.728142 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:42.728084    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:56.885318 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:56.885231    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:19:59.460180 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:59.460146    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-rr97k_45ffa13d-635a-46e8-8894-30902b9cf719/console-operator/2.log"
Sep 17 14:19:59.461170 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:19:59.461152    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-rr97k_45ffa13d-635a-46e8-8894-30902b9cf719/console-operator/2.log"
Sep 17 14:20:00.416274 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:00.416236    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:01.514228 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:01.514186    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9e4372d6-pcvg6"]
Sep 17 14:20:01.514900 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:01.514878    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="853a644f-4184-47ef-863e-c08ac7dbee38" containerName="revoke-keys"
Sep 17 14:20:01.515008 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:01.514902    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="853a644f-4184-47ef-863e-c08ac7dbee38" containerName="revoke-keys"
Sep 17 14:20:01.515062 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:01.515010    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="853a644f-4184-47ef-863e-c08ac7dbee38" containerName="revoke-keys"
Sep 17 14:20:01.519530 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:01.519510    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9e4372d6-pcvg6"
Sep 17 14:20:01.522358 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:01.522209    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ddwrn\""
Sep 17 14:20:01.526338 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:01.526275    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9e4372d6-pcvg6"]
Sep 17 14:20:01.647173 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:01.647129    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbc00c24-ed14-40ab-a73a-7ecc0bb7b1e6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-9e4372d6-pcvg6\" (UID: \"dbc00c24-ed14-40ab-a73a-7ecc0bb7b1e6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9e4372d6-pcvg6"
Sep 17 14:20:01.747876 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:01.747840    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbc00c24-ed14-40ab-a73a-7ecc0bb7b1e6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-9e4372d6-pcvg6\" (UID: \"dbc00c24-ed14-40ab-a73a-7ecc0bb7b1e6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9e4372d6-pcvg6"
Sep 17 14:20:01.748466 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:01.748444    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbc00c24-ed14-40ab-a73a-7ecc0bb7b1e6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-9e4372d6-pcvg6\" (UID: \"dbc00c24-ed14-40ab-a73a-7ecc0bb7b1e6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9e4372d6-pcvg6"
Sep 17 14:20:01.833359 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:01.833277    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9e4372d6-pcvg6"
Sep 17 14:20:01.943105 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:01.943041    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:02.108960 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:02.108929    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9e4372d6-pcvg6"]
Sep 17 14:20:02.110513 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:20:02.110489    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbc00c24_ed14_40ab_a73a_7ecc0bb7b1e6.slice/crio-3c75007135076a6e2cc08376156035c98bf413713fa21ec565a84d1736761984 WatchSource:0}: Error finding container 3c75007135076a6e2cc08376156035c98bf413713fa21ec565a84d1736761984: Status 404 returned error can't find the container with id 3c75007135076a6e2cc08376156035c98bf413713fa21ec565a84d1736761984
Sep 17 14:20:02.260520 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:02.260477    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9e4372d6-pcvg6" event={"ID":"dbc00c24-ed14-40ab-a73a-7ecc0bb7b1e6","Type":"ContainerStarted","Data":"3c75007135076a6e2cc08376156035c98bf413713fa21ec565a84d1736761984"}
Sep 17 14:20:02.746721 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:02.746686    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:03.267473 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:03.267438    2582 generic.go:356] "Generic (PLEG): container finished" podID="dbc00c24-ed14-40ab-a73a-7ecc0bb7b1e6" containerID="42541548cda2927085935dae6380265c0cbc41cf965e1123bb4d107414e29388" exitCode=0
Sep 17 14:20:03.267675 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:03.267473    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9e4372d6-pcvg6" event={"ID":"dbc00c24-ed14-40ab-a73a-7ecc0bb7b1e6","Type":"ContainerDied","Data":"42541548cda2927085935dae6380265c0cbc41cf965e1123bb4d107414e29388"}
Sep 17 14:20:03.544951 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:03.544914    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:04.434739 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:04.434707    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9e4372d6-pcvg6"
Sep 17 14:20:04.577104 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:04.577004    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbc00c24-ed14-40ab-a73a-7ecc0bb7b1e6-maas-api-service-ca\") pod \"dbc00c24-ed14-40ab-a73a-7ecc0bb7b1e6\" (UID: \"dbc00c24-ed14-40ab-a73a-7ecc0bb7b1e6\") "
Sep 17 14:20:04.577408 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:04.577382    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbc00c24-ed14-40ab-a73a-7ecc0bb7b1e6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dbc00c24-ed14-40ab-a73a-7ecc0bb7b1e6" (UID: "dbc00c24-ed14-40ab-a73a-7ecc0bb7b1e6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:20:04.577721 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:04.577696    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbc00c24-ed14-40ab-a73a-7ecc0bb7b1e6-maas-api-service-ca\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:20:04.678073 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:04.678030    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:05.040059 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:05.040027    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:05.280941 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:05.280903    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9e4372d6-pcvg6" event={"ID":"dbc00c24-ed14-40ab-a73a-7ecc0bb7b1e6","Type":"ContainerDied","Data":"3c75007135076a6e2cc08376156035c98bf413713fa21ec565a84d1736761984"}
Sep 17 14:20:05.280941 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:05.280933    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9e4372d6-pcvg6"
Sep 17 14:20:05.281154 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:05.280958    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3c75007135076a6e2cc08376156035c98bf413713fa21ec565a84d1736761984"
Sep 17 14:20:05.732157 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:05.732124    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:06.528130 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:06.528058    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:07.223697 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:07.223655    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:07.913581 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:07.913547    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:08.007145 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:08.007114    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:08.080585 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:08.080547    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9e4372d6-pcvg6"]
Sep 17 14:20:08.082676 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:08.082649    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:08.082830 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:08.082810    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9e4372d6-pcvg6"]
Sep 17 14:20:08.152914 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:08.152863    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:08.776814 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:08.776782    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:08.879716 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:08.879684    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:08.988724 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:08.988688    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:09.377319 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:09.377288    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:09.473284 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:09.473248    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:09.494557 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:09.494522    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbc00c24-ed14-40ab-a73a-7ecc0bb7b1e6" path="/var/lib/kubelet/pods/dbc00c24-ed14-40ab-a73a-7ecc0bb7b1e6/volumes"
Sep 17 14:20:10.246720 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:10.246687    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:11.131241 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:11.131202    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:11.654229 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:11.654163    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:11.750817 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:11.750779    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:12.131813 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:12.131776    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:12.834299 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:12.834260    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:13.589268 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:13.589233    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:13.657585 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:13.657544    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:13.850426 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:13.850329    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:14.056003 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:14.055966    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:14.220763 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:14.220731    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:14.650565 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:14.650479    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:15.228253 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:15.228215    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:16.177262 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:16.177227    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:16.850132 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:16.850099    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:18.694090 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:18.694054    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:28.174785 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:28.174744    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:36.120220 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:36.120181    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:36.348781 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:36.348741    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:37.279278 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:37.279238    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:37.436163 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:37.436126    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60f59148-gcv95"]
Sep 17 14:20:37.436642 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:37.436629    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbc00c24-ed14-40ab-a73a-7ecc0bb7b1e6" containerName="revoke-keys"
Sep 17 14:20:37.436692 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:37.436644    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbc00c24-ed14-40ab-a73a-7ecc0bb7b1e6" containerName="revoke-keys"
Sep 17 14:20:37.436746 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:37.436736    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbc00c24-ed14-40ab-a73a-7ecc0bb7b1e6" containerName="revoke-keys"
Sep 17 14:20:37.440182 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:37.440163    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60f59148-gcv95"
Sep 17 14:20:37.443198 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:37.443173    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ddwrn\""
Sep 17 14:20:37.451776 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:37.451748    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60f59148-gcv95"]
Sep 17 14:20:37.532483 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:37.532382    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12737858-bce1-4bd8-a500-11c8c085a5b7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-60f59148-gcv95\" (UID: \"12737858-bce1-4bd8-a500-11c8c085a5b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60f59148-gcv95"
Sep 17 14:20:37.634066 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:37.634026    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12737858-bce1-4bd8-a500-11c8c085a5b7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-60f59148-gcv95\" (UID: \"12737858-bce1-4bd8-a500-11c8c085a5b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60f59148-gcv95"
Sep 17 14:20:37.634878 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:37.634859    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12737858-bce1-4bd8-a500-11c8c085a5b7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-60f59148-gcv95\" (UID: \"12737858-bce1-4bd8-a500-11c8c085a5b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60f59148-gcv95"
Sep 17 14:20:37.751602 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:37.751562    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60f59148-gcv95"
Sep 17 14:20:37.945872 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:37.945832    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:38.037532 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:38.037501    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60f59148-gcv95"]
Sep 17 14:20:38.039643 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:20:38.039588    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12737858_bce1_4bd8_a500_11c8c085a5b7.slice/crio-e02ce4e3328b304bc72611bfaf34004110077110d5b23d24070d274ea038e5e6 WatchSource:0}: Error finding container e02ce4e3328b304bc72611bfaf34004110077110d5b23d24070d274ea038e5e6: Status 404 returned error can't find the container with id e02ce4e3328b304bc72611bfaf34004110077110d5b23d24070d274ea038e5e6
Sep 17 14:20:38.486839 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:38.486797    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60f59148-gcv95" event={"ID":"12737858-bce1-4bd8-a500-11c8c085a5b7","Type":"ContainerStarted","Data":"5d5d6f9f13da2ae499c4fffd447179e123f45d189ae73cb98f426b9111d2acd3"}
Sep 17 14:20:38.487214 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:38.486853    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60f59148-gcv95" event={"ID":"12737858-bce1-4bd8-a500-11c8c085a5b7","Type":"ContainerStarted","Data":"e02ce4e3328b304bc72611bfaf34004110077110d5b23d24070d274ea038e5e6"}
Sep 17 14:20:38.505356 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:38.505303    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60f59148-gcv95" podStartSLOduration=1.5052856399999999 podStartE2EDuration="1.50528564s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:20:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:20:38.503978363 +0000 UTC m=+939.505521839" watchObservedRunningTime="2026-09-17 14:20:38.50528564 +0000 UTC m=+939.506829139"
Sep 17 14:20:39.209076 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:39.209042    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:39.447704 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:39.447659    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:39.495162 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:39.495068    2582 generic.go:356] "Generic (PLEG): container finished" podID="12737858-bce1-4bd8-a500-11c8c085a5b7" containerID="5d5d6f9f13da2ae499c4fffd447179e123f45d189ae73cb98f426b9111d2acd3" exitCode=0
Sep 17 14:20:39.495561 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:39.495155    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60f59148-gcv95" event={"ID":"12737858-bce1-4bd8-a500-11c8c085a5b7","Type":"ContainerDied","Data":"5d5d6f9f13da2ae499c4fffd447179e123f45d189ae73cb98f426b9111d2acd3"}
Sep 17 14:20:40.145247 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:40.145209    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:40.650644 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:40.650602    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60f59148-gcv95"
Sep 17 14:20:40.665781 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:40.665722    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12737858-bce1-4bd8-a500-11c8c085a5b7-maas-api-service-ca\") pod \"12737858-bce1-4bd8-a500-11c8c085a5b7\" (UID: \"12737858-bce1-4bd8-a500-11c8c085a5b7\") "
Sep 17 14:20:40.666488 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:40.666451    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12737858-bce1-4bd8-a500-11c8c085a5b7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "12737858-bce1-4bd8-a500-11c8c085a5b7" (UID: "12737858-bce1-4bd8-a500-11c8c085a5b7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:20:40.767116 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:40.767015    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12737858-bce1-4bd8-a500-11c8c085a5b7-maas-api-service-ca\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:20:41.471550 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:41.471515    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:41.508052 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:41.508025    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60f59148-gcv95"
Sep 17 14:20:41.508190 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:41.508054    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60f59148-gcv95" event={"ID":"12737858-bce1-4bd8-a500-11c8c085a5b7","Type":"ContainerDied","Data":"e02ce4e3328b304bc72611bfaf34004110077110d5b23d24070d274ea038e5e6"}
Sep 17 14:20:41.508190 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:41.508095    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e02ce4e3328b304bc72611bfaf34004110077110d5b23d24070d274ea038e5e6"
Sep 17 14:20:41.673692 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:41.673661    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:42.344728 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:42.344694    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:42.931580 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:42.931539    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:43.032734 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:43.032692    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:43.112157 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:43.112111    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60f59148-gcv95"]
Sep 17 14:20:43.114254 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:43.114231    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:43.114706 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:43.114687    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60f59148-gcv95"]
Sep 17 14:20:43.174293 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:43.174267    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:43.493982 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:43.493938    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12737858-bce1-4bd8-a500-11c8c085a5b7" path="/var/lib/kubelet/pods/12737858-bce1-4bd8-a500-11c8c085a5b7/volumes"
Sep 17 14:20:43.599481 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:43.599451    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:44.285182 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:44.285133    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:44.404807 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:44.404773    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:44.793040 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:44.793006    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:44.884559 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:44.884524    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:45.042382 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:45.042346    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj"]
Sep 17 14:20:45.042983 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:45.042962    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12737858-bce1-4bd8-a500-11c8c085a5b7" containerName="revoke-keys"
Sep 17 14:20:45.042983 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:45.042983    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="12737858-bce1-4bd8-a500-11c8c085a5b7" containerName="revoke-keys"
Sep 17 14:20:45.043196 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:45.043152    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="12737858-bce1-4bd8-a500-11c8c085a5b7" containerName="revoke-keys"
Sep 17 14:20:45.046720 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:45.046696    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj"
Sep 17 14:20:45.049129 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:45.049100    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-99500ae5-openshift-default-dockercfg-65jst\""
Sep 17 14:20:45.059375 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:45.059347    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj"]
Sep 17 14:20:45.063258 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:45.063217    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:45.111849 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:45.111813    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qld6\" (UniqueName: \"kubernetes.io/projected/eb2a1136-87dc-4692-9e19-cb692f34948a-kube-api-access-4qld6\") pod \"e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj\" (UID: \"eb2a1136-87dc-4692-9e19-cb692f34948a\") " pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj"
Sep 17 14:20:45.111849 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:45.111855    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb2a1136-87dc-4692-9e19-cb692f34948a-credential-socket\") pod \"e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj\" (UID: \"eb2a1136-87dc-4692-9e19-cb692f34948a\") " pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj"
Sep 17 14:20:45.112091 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:45.111873    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb2a1136-87dc-4692-9e19-cb692f34948a-istiod-ca-cert\") pod \"e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj\" (UID: \"eb2a1136-87dc-4692-9e19-cb692f34948a\") " pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj"
Sep 17 14:20:45.112091 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:45.111994    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb2a1136-87dc-4692-9e19-cb692f34948a-workload-certs\") pod \"e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj\" (UID: \"eb2a1136-87dc-4692-9e19-cb692f34948a\") " pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj"
Sep 17 14:20:45.112091 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:45.112070    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb2a1136-87dc-4692-9e19-cb692f34948a-istio-token\") pod \"e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj\" (UID: \"eb2a1136-87dc-4692-9e19-cb692f34948a\") " pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj"
Sep 17 14:20:45.112207 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:45.112112    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb2a1136-87dc-4692-9e19-cb692f34948a-istio-data\") pod \"e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj\" (UID: \"eb2a1136-87dc-4692-9e19-cb692f34948a\") " pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj"
Sep 17 14:20:45.112207 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:45.112139    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb2a1136-87dc-4692-9e19-cb692f34948a-istio-podinfo\") pod \"e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj\" (UID: \"eb2a1136-87dc-4692-9e19-cb692f34948a\") " pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj"
Sep 17 14:20:45.112207 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:45.112169    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb2a1136-87dc-4692-9e19-cb692f34948a-istio-envoy\") pod \"e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj\" (UID: \"eb2a1136-87dc-4692-9e19-cb692f34948a\") " pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj"
Sep 17 14:20:45.112313 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:45.112222    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb2a1136-87dc-4692-9e19-cb692f34948a-workload-socket\") pod \"e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj\" (UID: \"eb2a1136-87dc-4692-9e19-cb692f34948a\") " pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj"
Sep 17 14:20:45.213823 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:45.213782    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb2a1136-87dc-4692-9e19-cb692f34948a-istio-envoy\") pod \"e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj\" (UID: \"eb2a1136-87dc-4692-9e19-cb692f34948a\") " pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj"
Sep 17 14:20:45.214001 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:45.213853    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb2a1136-87dc-4692-9e19-cb692f34948a-workload-socket\") pod \"e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj\" (UID: \"eb2a1136-87dc-4692-9e19-cb692f34948a\") " pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj"
Sep 17 14:20:45.214001 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:45.213911    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qld6\" (UniqueName: \"kubernetes.io/projected/eb2a1136-87dc-4692-9e19-cb692f34948a-kube-api-access-4qld6\") pod \"e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj\" (UID: \"eb2a1136-87dc-4692-9e19-cb692f34948a\") " pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj"
Sep 17 14:20:45.214001 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:45.213940    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb2a1136-87dc-4692-9e19-cb692f34948a-credential-socket\") pod \"e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj\" (UID: \"eb2a1136-87dc-4692-9e19-cb692f34948a\") " pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj"
Sep 17 14:20:45.214001 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:45.213967    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb2a1136-87dc-4692-9e19-cb692f34948a-istiod-ca-cert\") pod \"e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj\" (UID: \"eb2a1136-87dc-4692-9e19-cb692f34948a\") " pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj"
Sep 17 14:20:45.214135 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:45.214098    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb2a1136-87dc-4692-9e19-cb692f34948a-workload-certs\") pod \"e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj\" (UID: \"eb2a1136-87dc-4692-9e19-cb692f34948a\") " pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj"
Sep 17 14:20:45.214187 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:45.214171    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb2a1136-87dc-4692-9e19-cb692f34948a-istio-token\") pod \"e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj\" (UID: \"eb2a1136-87dc-4692-9e19-cb692f34948a\") " pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj"
Sep 17 14:20:45.214241 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:45.214228    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb2a1136-87dc-4692-9e19-cb692f34948a-istio-data\") pod \"e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj\" (UID: \"eb2a1136-87dc-4692-9e19-cb692f34948a\") " pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj"
Sep 17 14:20:45.214295 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:45.214266    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb2a1136-87dc-4692-9e19-cb692f34948a-istio-podinfo\") pod \"e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj\" (UID: \"eb2a1136-87dc-4692-9e19-cb692f34948a\") " pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj"
Sep 17 14:20:45.214645 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:45.214363    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb2a1136-87dc-4692-9e19-cb692f34948a-credential-socket\") pod \"e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj\" (UID: \"eb2a1136-87dc-4692-9e19-cb692f34948a\") " pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj"
Sep 17 14:20:45.214645 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:45.214415    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb2a1136-87dc-4692-9e19-cb692f34948a-workload-socket\") pod \"e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj\" (UID: \"eb2a1136-87dc-4692-9e19-cb692f34948a\") " pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj"
Sep 17 14:20:45.214645 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:45.214559    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb2a1136-87dc-4692-9e19-cb692f34948a-workload-certs\") pod \"e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj\" (UID: \"eb2a1136-87dc-4692-9e19-cb692f34948a\") " pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj"
Sep 17 14:20:45.214842 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:45.214694    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb2a1136-87dc-4692-9e19-cb692f34948a-istio-data\") pod \"e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj\" (UID: \"eb2a1136-87dc-4692-9e19-cb692f34948a\") " pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj"
Sep 17 14:20:45.214842 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:45.214769    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb2a1136-87dc-4692-9e19-cb692f34948a-istiod-ca-cert\") pod \"e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj\" (UID: \"eb2a1136-87dc-4692-9e19-cb692f34948a\") " pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj"
Sep 17 14:20:45.216214 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:45.216195    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb2a1136-87dc-4692-9e19-cb692f34948a-istio-envoy\") pod \"e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj\" (UID: \"eb2a1136-87dc-4692-9e19-cb692f34948a\") " pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj"
Sep 17 14:20:45.216578 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:45.216560    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb2a1136-87dc-4692-9e19-cb692f34948a-istio-podinfo\") pod \"e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj\" (UID: \"eb2a1136-87dc-4692-9e19-cb692f34948a\") " pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj"
Sep 17 14:20:45.222791 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:45.222766    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb2a1136-87dc-4692-9e19-cb692f34948a-istio-token\") pod \"e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj\" (UID: \"eb2a1136-87dc-4692-9e19-cb692f34948a\") " pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj"
Sep 17 14:20:45.222932 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:45.222912    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qld6\" (UniqueName: \"kubernetes.io/projected/eb2a1136-87dc-4692-9e19-cb692f34948a-kube-api-access-4qld6\") pod \"e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj\" (UID: \"eb2a1136-87dc-4692-9e19-cb692f34948a\") " pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj"
Sep 17 14:20:45.231291 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:45.231263    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:45.331559 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:45.331470    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:45.358600 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:45.358543    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj"
Sep 17 14:20:45.635066 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:45.635030    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj"]
Sep 17 14:20:45.636968 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:20:45.636939    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb2a1136_87dc_4692_9e19_cb692f34948a.slice/crio-ab4f57607e95a8b4727be06942654f101aa6d5a365b4bc9173554a3f561eaa42 WatchSource:0}: Error finding container ab4f57607e95a8b4727be06942654f101aa6d5a365b4bc9173554a3f561eaa42: Status 404 returned error can't find the container with id ab4f57607e95a8b4727be06942654f101aa6d5a365b4bc9173554a3f561eaa42
Sep 17 14:20:45.639320 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:45.639289    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:20:45.639380 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:45.639365    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:20:45.639417 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:45.639394    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:20:45.931978 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:45.931880    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:46.538490 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:46.538452    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj" event={"ID":"eb2a1136-87dc-4692-9e19-cb692f34948a","Type":"ContainerStarted","Data":"a9ab067c8a55aad076f883ba6b12c0039c549a6e8ff75980e8b7e72a4f97e689"}
Sep 17 14:20:46.538913 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:46.538513    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj" event={"ID":"eb2a1136-87dc-4692-9e19-cb692f34948a","Type":"ContainerStarted","Data":"ab4f57607e95a8b4727be06942654f101aa6d5a365b4bc9173554a3f561eaa42"}
Sep 17 14:20:46.564161 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:46.564109    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj" podStartSLOduration=1.564090153 podStartE2EDuration="1.564090153s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:20:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:20:46.561005085 +0000 UTC m=+947.562548558" watchObservedRunningTime="2026-09-17 14:20:46.564090153 +0000 UTC m=+947.565633628"
Sep 17 14:20:46.746662 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:46.746593    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:47.358938 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:47.358893    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj"
Sep 17 14:20:47.364547 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:47.364522    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj"
Sep 17 14:20:47.545757 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:47.545724    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj"
Sep 17 14:20:47.546678 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:47.546655    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj"
Sep 17 14:20:47.641703 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:47.641593    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:48.727350 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:48.727298    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:48.829721 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:48.829680    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:49.290153 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:49.290106    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:50.636800 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:50.636765    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:50.736322 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:50.736287    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:51.441983 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:51.441948    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:52.139791 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:52.139750    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:52.899700 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:52.899655    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:52.966918 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:52.966870    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:53.799893 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:53.799849    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-3f6f32-g6b2b"]
Sep 17 14:20:53.803929 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:53.803905    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-3f6f32-g6b2b"
Sep 17 14:20:53.806287 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:53.806267    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ddwrn\""
Sep 17 14:20:53.812957 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:53.812930    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-3f6f32-g6b2b"]
Sep 17 14:20:53.897772 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:53.897740    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c00e7e21-d93b-407d-a2be-819c537e02bd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-3f6f32-g6b2b\" (UID: \"c00e7e21-d93b-407d-a2be-819c537e02bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-3f6f32-g6b2b"
Sep 17 14:20:53.999337 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:53.999290    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c00e7e21-d93b-407d-a2be-819c537e02bd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-3f6f32-g6b2b\" (UID: \"c00e7e21-d93b-407d-a2be-819c537e02bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-3f6f32-g6b2b"
Sep 17 14:20:53.999966 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:53.999944    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c00e7e21-d93b-407d-a2be-819c537e02bd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-3f6f32-g6b2b\" (UID: \"c00e7e21-d93b-407d-a2be-819c537e02bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-3f6f32-g6b2b"
Sep 17 14:20:54.117454 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:54.117367    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-3f6f32-g6b2b"
Sep 17 14:20:54.189335 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:54.189303    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:54.390323 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:54.390293    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-3f6f32-g6b2b"]
Sep 17 14:20:54.392117 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:20:54.392091    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc00e7e21_d93b_407d_a2be_819c537e02bd.slice/crio-744ffe8b0a5d102ffb77699495f196645488fabf4474f6b4dfb1b37154514a63 WatchSource:0}: Error finding container 744ffe8b0a5d102ffb77699495f196645488fabf4474f6b4dfb1b37154514a63: Status 404 returned error can't find the container with id 744ffe8b0a5d102ffb77699495f196645488fabf4474f6b4dfb1b37154514a63
Sep 17 14:20:54.588963 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:54.588926    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-3f6f32-g6b2b" event={"ID":"c00e7e21-d93b-407d-a2be-819c537e02bd","Type":"ContainerStarted","Data":"744ffe8b0a5d102ffb77699495f196645488fabf4474f6b4dfb1b37154514a63"}
Sep 17 14:20:54.920043 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:54.919953    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:55.596551 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:55.596450    2582 generic.go:356] "Generic (PLEG): container finished" podID="c00e7e21-d93b-407d-a2be-819c537e02bd" containerID="06f8c189b5755f666b171414759c1def338f76c148f209721d4f676dc2ec2e15" exitCode=0
Sep 17 14:20:55.596551 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:55.596508    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-3f6f32-g6b2b" event={"ID":"c00e7e21-d93b-407d-a2be-819c537e02bd","Type":"ContainerDied","Data":"06f8c189b5755f666b171414759c1def338f76c148f209721d4f676dc2ec2e15"}
Sep 17 14:20:55.728867 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:55.728773    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:56.558155 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:56.558118    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:56.761475 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:56.761449    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-3f6f32-g6b2b"
Sep 17 14:20:56.830022 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:56.829933    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c00e7e21-d93b-407d-a2be-819c537e02bd-maas-api-service-ca\") pod \"c00e7e21-d93b-407d-a2be-819c537e02bd\" (UID: \"c00e7e21-d93b-407d-a2be-819c537e02bd\") "
Sep 17 14:20:56.830362 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:56.830334    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c00e7e21-d93b-407d-a2be-819c537e02bd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c00e7e21-d93b-407d-a2be-819c537e02bd" (UID: "c00e7e21-d93b-407d-a2be-819c537e02bd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:20:56.931522 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:56.931490    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c00e7e21-d93b-407d-a2be-819c537e02bd-maas-api-service-ca\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:20:57.225484 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:57.225442    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:57.609210 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:57.609184    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-3f6f32-g6b2b"
Sep 17 14:20:57.609647 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:57.609212    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-3f6f32-g6b2b" event={"ID":"c00e7e21-d93b-407d-a2be-819c537e02bd","Type":"ContainerDied","Data":"744ffe8b0a5d102ffb77699495f196645488fabf4474f6b4dfb1b37154514a63"}
Sep 17 14:20:57.609647 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:57.609252    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="744ffe8b0a5d102ffb77699495f196645488fabf4474f6b4dfb1b37154514a63"
Sep 17 14:20:58.070296 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:58.070231    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:20:58.735876 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:20:58.735838    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:21:00.389273 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:00.389233    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:21:01.106632 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:01.106562    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:21:04.311705 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:04.311669    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-3f6f32-796794d758-5sscr"]
Sep 17 14:21:04.312483 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:04.312432    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-3f6f32-796794d758-5sscr" podUID="e09c655f-30c2-4939-a73b-03ecdf25ff5a" containerName="payload-pre-processing" containerID="cri-o://036b8edc50a8ab8611ffbc03ae61fb2b9a0b5227113a5c9216c1d5771e14a99b" gracePeriod=30
Sep 17 14:21:04.455337 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:04.455276    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-3f6f32-g6b2b"]
Sep 17 14:21:04.460522 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:04.460492    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-3f6f32-g6b2b"]
Sep 17 14:21:04.669058 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:04.669018    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-3f6f32-796794d758-5sscr"
Sep 17 14:21:04.671515 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:04.671490    2582 generic.go:356] "Generic (PLEG): container finished" podID="e09c655f-30c2-4939-a73b-03ecdf25ff5a" containerID="036b8edc50a8ab8611ffbc03ae61fb2b9a0b5227113a5c9216c1d5771e14a99b" exitCode=0
Sep 17 14:21:04.671668 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:04.671521    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-3f6f32-796794d758-5sscr"
Sep 17 14:21:04.671668 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:04.671570    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-3f6f32-796794d758-5sscr" event={"ID":"e09c655f-30c2-4939-a73b-03ecdf25ff5a","Type":"ContainerDied","Data":"036b8edc50a8ab8611ffbc03ae61fb2b9a0b5227113a5c9216c1d5771e14a99b"}
Sep 17 14:21:04.671668 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:04.671637    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-3f6f32-796794d758-5sscr" event={"ID":"e09c655f-30c2-4939-a73b-03ecdf25ff5a","Type":"ContainerDied","Data":"85b09fa7ff4a65a6cff832fd72553931516937d0ae3cedb89f9092faccef3685"}
Sep 17 14:21:04.671668 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:04.671659    2582 scope.go:117] "RemoveContainer" containerID="036b8edc50a8ab8611ffbc03ae61fb2b9a0b5227113a5c9216c1d5771e14a99b"
Sep 17 14:21:04.681415 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:04.681399    2582 scope.go:117] "RemoveContainer" containerID="036b8edc50a8ab8611ffbc03ae61fb2b9a0b5227113a5c9216c1d5771e14a99b"
Sep 17 14:21:04.681710 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:21:04.681687    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"036b8edc50a8ab8611ffbc03ae61fb2b9a0b5227113a5c9216c1d5771e14a99b\": container with ID starting with 036b8edc50a8ab8611ffbc03ae61fb2b9a0b5227113a5c9216c1d5771e14a99b not found: ID does not exist" containerID="036b8edc50a8ab8611ffbc03ae61fb2b9a0b5227113a5c9216c1d5771e14a99b"
Sep 17 14:21:04.681783 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:04.681720    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"036b8edc50a8ab8611ffbc03ae61fb2b9a0b5227113a5c9216c1d5771e14a99b"} err="failed to get container status \"036b8edc50a8ab8611ffbc03ae61fb2b9a0b5227113a5c9216c1d5771e14a99b\": rpc error: code = NotFound desc = could not find container \"036b8edc50a8ab8611ffbc03ae61fb2b9a0b5227113a5c9216c1d5771e14a99b\": container with ID starting with 036b8edc50a8ab8611ffbc03ae61fb2b9a0b5227113a5c9216c1d5771e14a99b not found: ID does not exist"
Sep 17 14:21:04.713166 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:04.713126    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e09c655f-30c2-4939-a73b-03ecdf25ff5a-plugins-config-volume\") pod \"e09c655f-30c2-4939-a73b-03ecdf25ff5a\" (UID: \"e09c655f-30c2-4939-a73b-03ecdf25ff5a\") "
Sep 17 14:21:04.713330 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:04.713211    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-txwgq\" (UniqueName: \"kubernetes.io/projected/e09c655f-30c2-4939-a73b-03ecdf25ff5a-kube-api-access-txwgq\") pod \"e09c655f-30c2-4939-a73b-03ecdf25ff5a\" (UID: \"e09c655f-30c2-4939-a73b-03ecdf25ff5a\") "
Sep 17 14:21:04.713580 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:04.713495    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e09c655f-30c2-4939-a73b-03ecdf25ff5a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e09c655f-30c2-4939-a73b-03ecdf25ff5a" (UID: "e09c655f-30c2-4939-a73b-03ecdf25ff5a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:21:04.713749 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:04.713641    2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e09c655f-30c2-4939-a73b-03ecdf25ff5a-plugins-config-volume\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:21:04.716657 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:04.715775    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e09c655f-30c2-4939-a73b-03ecdf25ff5a-kube-api-access-txwgq" (OuterVolumeSpecName: "kube-api-access-txwgq") pod "e09c655f-30c2-4939-a73b-03ecdf25ff5a" (UID: "e09c655f-30c2-4939-a73b-03ecdf25ff5a"). InnerVolumeSpecName "kube-api-access-txwgq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:21:04.815008 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:04.814971    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txwgq\" (UniqueName: \"kubernetes.io/projected/e09c655f-30c2-4939-a73b-03ecdf25ff5a-kube-api-access-txwgq\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:21:04.995669 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:04.995639    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-3f6f32-796794d758-5sscr"]
Sep 17 14:21:05.000759 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:05.000736    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-3f6f32-796794d758-5sscr"]
Sep 17 14:21:05.102938 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:05.102903    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:21:05.203934 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:05.203900    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:21:05.495134 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:05.495095    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c00e7e21-d93b-407d-a2be-819c537e02bd" path="/var/lib/kubelet/pods/c00e7e21-d93b-407d-a2be-819c537e02bd/volumes"
Sep 17 14:21:05.495509 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:05.495473    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e09c655f-30c2-4939-a73b-03ecdf25ff5a" path="/var/lib/kubelet/pods/e09c655f-30c2-4939-a73b-03ecdf25ff5a/volumes"
Sep 17 14:21:05.646944 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:05.646904    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:21:06.323054 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:06.323019    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:21:07.168019 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:07.167983    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:21:07.658734 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:07.658694    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-99500ae5-2kdhh"]
Sep 17 14:21:07.659186 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:07.659171    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e09c655f-30c2-4939-a73b-03ecdf25ff5a" containerName="payload-pre-processing"
Sep 17 14:21:07.659186 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:07.659185    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e09c655f-30c2-4939-a73b-03ecdf25ff5a" containerName="payload-pre-processing"
Sep 17 14:21:07.659325 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:07.659205    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c00e7e21-d93b-407d-a2be-819c537e02bd" containerName="revoke-keys"
Sep 17 14:21:07.659325 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:07.659211    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c00e7e21-d93b-407d-a2be-819c537e02bd" containerName="revoke-keys"
Sep 17 14:21:07.659325 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:07.659287    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c00e7e21-d93b-407d-a2be-819c537e02bd" containerName="revoke-keys"
Sep 17 14:21:07.659325 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:07.659297    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e09c655f-30c2-4939-a73b-03ecdf25ff5a" containerName="payload-pre-processing"
Sep 17 14:21:07.670118 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:07.669693    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-99500ae5-2kdhh"
Sep 17 14:21:07.673243 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:07.673207    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ddwrn\""
Sep 17 14:21:07.675121 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:07.675088    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-99500ae5-2kdhh"]
Sep 17 14:21:07.751103 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:07.751065    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2329a73f-b523-4312-a5d2-4fd886fcd46a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-99500ae5-2kdhh\" (UID: \"2329a73f-b523-4312-a5d2-4fd886fcd46a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-99500ae5-2kdhh"
Sep 17 14:21:07.852758 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:07.852716    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2329a73f-b523-4312-a5d2-4fd886fcd46a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-99500ae5-2kdhh\" (UID: \"2329a73f-b523-4312-a5d2-4fd886fcd46a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-99500ae5-2kdhh"
Sep 17 14:21:07.853401 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:07.853374    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2329a73f-b523-4312-a5d2-4fd886fcd46a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-99500ae5-2kdhh\" (UID: \"2329a73f-b523-4312-a5d2-4fd886fcd46a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-99500ae5-2kdhh"
Sep 17 14:21:07.984203 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:07.984162    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-99500ae5-2kdhh"
Sep 17 14:21:08.033294 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:08.033254    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:21:08.262318 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:08.262291    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-99500ae5-2kdhh"]
Sep 17 14:21:08.263964 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:21:08.263938    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2329a73f_b523_4312_a5d2_4fd886fcd46a.slice/crio-285a0541c6141a3a6e01c1ecb0ae2ef4e52b6a7f2a6f83a87e771540e5210887 WatchSource:0}: Error finding container 285a0541c6141a3a6e01c1ecb0ae2ef4e52b6a7f2a6f83a87e771540e5210887: Status 404 returned error can't find the container with id 285a0541c6141a3a6e01c1ecb0ae2ef4e52b6a7f2a6f83a87e771540e5210887
Sep 17 14:21:08.701803 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:08.701748    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-99500ae5-2kdhh" event={"ID":"2329a73f-b523-4312-a5d2-4fd886fcd46a","Type":"ContainerStarted","Data":"76c92fa7d1c81ab9f4d11d51fb6d7590b59a4ef4a1293187ca2a472efee17ab8"}
Sep 17 14:21:08.701976 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:08.701829    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-99500ae5-2kdhh" event={"ID":"2329a73f-b523-4312-a5d2-4fd886fcd46a","Type":"ContainerStarted","Data":"285a0541c6141a3a6e01c1ecb0ae2ef4e52b6a7f2a6f83a87e771540e5210887"}
Sep 17 14:21:08.718493 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:08.718427    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-99500ae5-2kdhh" podStartSLOduration=1.7184073199999998 podStartE2EDuration="1.71840732s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:21:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:21:08.717383675 +0000 UTC m=+969.718927149" watchObservedRunningTime="2026-09-17 14:21:08.71840732 +0000 UTC m=+969.719950795"
Sep 17 14:21:08.819446 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:08.819357    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:21:09.530379 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:09.530292    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:21:09.714057 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:09.714015    2582 generic.go:356] "Generic (PLEG): container finished" podID="2329a73f-b523-4312-a5d2-4fd886fcd46a" containerID="76c92fa7d1c81ab9f4d11d51fb6d7590b59a4ef4a1293187ca2a472efee17ab8" exitCode=0
Sep 17 14:21:09.714250 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:09.714173    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-99500ae5-2kdhh" event={"ID":"2329a73f-b523-4312-a5d2-4fd886fcd46a","Type":"ContainerDied","Data":"76c92fa7d1c81ab9f4d11d51fb6d7590b59a4ef4a1293187ca2a472efee17ab8"}
Sep 17 14:21:09.910388 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:09.910309    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:21:10.270238 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:10.270204    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:21:10.908360 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:10.908323    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-99500ae5-2kdhh"
Sep 17 14:21:10.993640 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:10.993512    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2329a73f-b523-4312-a5d2-4fd886fcd46a-maas-api-service-ca\") pod \"2329a73f-b523-4312-a5d2-4fd886fcd46a\" (UID: \"2329a73f-b523-4312-a5d2-4fd886fcd46a\") "
Sep 17 14:21:10.994019 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:10.993986    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2329a73f-b523-4312-a5d2-4fd886fcd46a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2329a73f-b523-4312-a5d2-4fd886fcd46a" (UID: "2329a73f-b523-4312-a5d2-4fd886fcd46a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:21:11.064509 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:11.064473    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:21:11.095271 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:11.095231    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2329a73f-b523-4312-a5d2-4fd886fcd46a-maas-api-service-ca\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:21:11.728491 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:11.728455    2582 generic.go:356] "Generic (PLEG): container finished" podID="eb2a1136-87dc-4692-9e19-cb692f34948a" containerID="a9ab067c8a55aad076f883ba6b12c0039c549a6e8ff75980e8b7e72a4f97e689" exitCode=0
Sep 17 14:21:11.728782 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:11.728556    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj" event={"ID":"eb2a1136-87dc-4692-9e19-cb692f34948a","Type":"ContainerDied","Data":"a9ab067c8a55aad076f883ba6b12c0039c549a6e8ff75980e8b7e72a4f97e689"}
Sep 17 14:21:11.729131 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:11.729079    2582 scope.go:117] "RemoveContainer" containerID="a9ab067c8a55aad076f883ba6b12c0039c549a6e8ff75980e8b7e72a4f97e689"
Sep 17 14:21:11.730507 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:11.730473    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:21:11.730652 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:11.730541    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-99500ae5-2kdhh"
Sep 17 14:21:11.730652 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:11.730558    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:21:11.730652 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:11.730588    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-99500ae5-2kdhh" event={"ID":"2329a73f-b523-4312-a5d2-4fd886fcd46a","Type":"ContainerDied","Data":"285a0541c6141a3a6e01c1ecb0ae2ef4e52b6a7f2a6f83a87e771540e5210887"}
Sep 17 14:21:11.730652 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:11.730622    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:21:11.730902 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:11.730658    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="285a0541c6141a3a6e01c1ecb0ae2ef4e52b6a7f2a6f83a87e771540e5210887"
Sep 17 14:21:11.733653 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:11.733629    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:21:12.423729 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:12.423691    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:21:12.756508 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:12.756463    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj" event={"ID":"eb2a1136-87dc-4692-9e19-cb692f34948a","Type":"ContainerStarted","Data":"90f301fdb26acafb7fc06cdd48329a3f9468d2bf76178b4c11f1c835e4502a20"}
Sep 17 14:21:13.110288 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:13.110190    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:21:13.358811 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:13.358769    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj"
Sep 17 14:21:13.360173 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:13.360145    2582 patch_prober.go:28] interesting pod/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Sep 17 14:21:13.360323 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:13.360196    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj" podUID="eb2a1136-87dc-4692-9e19-cb692f34948a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Sep 17 14:21:14.359055 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:14.359016    2582 patch_prober.go:28] interesting pod/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Sep 17 14:21:14.359438 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:14.359079    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj" podUID="eb2a1136-87dc-4692-9e19-cb692f34948a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Sep 17 14:21:14.456320 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:14.456283    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:21:15.359308 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:15.359267    2582 patch_prober.go:28] interesting pod/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Sep 17 14:21:15.359719 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:15.359325    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj" podUID="eb2a1136-87dc-4692-9e19-cb692f34948a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Sep 17 14:21:16.359095 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:16.359058    2582 patch_prober.go:28] interesting pod/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Sep 17 14:21:16.359275 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:16.359115    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj" podUID="eb2a1136-87dc-4692-9e19-cb692f34948a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Sep 17 14:21:17.359946 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:17.359903    2582 patch_prober.go:28] interesting pod/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Sep 17 14:21:17.360339 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:17.359970    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj" podUID="eb2a1136-87dc-4692-9e19-cb692f34948a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Sep 17 14:21:17.545851 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:17.545809    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj"
Sep 17 14:21:18.197106 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:18.197071    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:21:18.267036 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:18.266997    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:21:18.345987 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:18.345938    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:21:18.363742 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:18.363707    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj"
Sep 17 14:21:18.364810 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:18.364788    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj"
Sep 17 14:21:18.392797 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:18.392766    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-99500ae5-2kdhh"]
Sep 17 14:21:18.397822 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:18.397796    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-99500ae5-2kdhh"]
Sep 17 14:21:18.432475 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:18.432441    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:21:18.542571 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:18.542488    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj"]
Sep 17 14:21:18.553481 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:18.553449    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:21:18.794552 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:18.794459    2582 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj" secret="" err="secret \"e2e-derive-99500ae5-openshift-default-dockercfg-65jst\" not found"
Sep 17 14:21:18.916603 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:18.916567    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:21:19.273342 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:19.273311    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:21:19.370416 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:19.370380    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:21:19.495117 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:19.495083    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2329a73f-b523-4312-a5d2-4fd886fcd46a" path="/var/lib/kubelet/pods/2329a73f-b523-4312-a5d2-4fd886fcd46a/volumes"
Sep 17 14:21:19.585093 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:19.585010    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:21:19.799922 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:19.799882    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj" podUID="eb2a1136-87dc-4692-9e19-cb692f34948a" containerName="istio-proxy" containerID="cri-o://90f301fdb26acafb7fc06cdd48329a3f9468d2bf76178b4c11f1c835e4502a20" gracePeriod=30
Sep 17 14:21:19.879944 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:19.879849    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:21:20.645304 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:20.645258    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:21:20.941141 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:20.941051    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:21:21.352567 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:21.352523    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:21:22.015255 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:22.015155    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:21:23.151105 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:23.151063    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:21:25.069353 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:25.069286    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj"
Sep 17 14:21:25.149671 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:25.149629    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb2a1136-87dc-4692-9e19-cb692f34948a-istio-data\") pod \"eb2a1136-87dc-4692-9e19-cb692f34948a\" (UID: \"eb2a1136-87dc-4692-9e19-cb692f34948a\") "
Sep 17 14:21:25.149856 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:25.149741    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4qld6\" (UniqueName: \"kubernetes.io/projected/eb2a1136-87dc-4692-9e19-cb692f34948a-kube-api-access-4qld6\") pod \"eb2a1136-87dc-4692-9e19-cb692f34948a\" (UID: \"eb2a1136-87dc-4692-9e19-cb692f34948a\") "
Sep 17 14:21:25.149856 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:25.149772    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb2a1136-87dc-4692-9e19-cb692f34948a-workload-socket\") pod \"eb2a1136-87dc-4692-9e19-cb692f34948a\" (UID: \"eb2a1136-87dc-4692-9e19-cb692f34948a\") "
Sep 17 14:21:25.149856 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:25.149812    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb2a1136-87dc-4692-9e19-cb692f34948a-istio-podinfo\") pod \"eb2a1136-87dc-4692-9e19-cb692f34948a\" (UID: \"eb2a1136-87dc-4692-9e19-cb692f34948a\") "
Sep 17 14:21:25.149856 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:25.149834    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb2a1136-87dc-4692-9e19-cb692f34948a-istiod-ca-cert\") pod \"eb2a1136-87dc-4692-9e19-cb692f34948a\" (UID: \"eb2a1136-87dc-4692-9e19-cb692f34948a\") "
Sep 17 14:21:25.150085 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:25.149852    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb2a1136-87dc-4692-9e19-cb692f34948a-istio-data" (OuterVolumeSpecName: "istio-data") pod "eb2a1136-87dc-4692-9e19-cb692f34948a" (UID: "eb2a1136-87dc-4692-9e19-cb692f34948a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:21:25.150085 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:25.150058    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb2a1136-87dc-4692-9e19-cb692f34948a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "eb2a1136-87dc-4692-9e19-cb692f34948a" (UID: "eb2a1136-87dc-4692-9e19-cb692f34948a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:21:25.150201 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:25.150121    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb2a1136-87dc-4692-9e19-cb692f34948a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "eb2a1136-87dc-4692-9e19-cb692f34948a" (UID: "eb2a1136-87dc-4692-9e19-cb692f34948a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:21:25.150201 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:25.150159    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb2a1136-87dc-4692-9e19-cb692f34948a-workload-certs\") pod \"eb2a1136-87dc-4692-9e19-cb692f34948a\" (UID: \"eb2a1136-87dc-4692-9e19-cb692f34948a\") "
Sep 17 14:21:25.150308 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:25.150203    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb2a1136-87dc-4692-9e19-cb692f34948a-istio-token\") pod \"eb2a1136-87dc-4692-9e19-cb692f34948a\" (UID: \"eb2a1136-87dc-4692-9e19-cb692f34948a\") "
Sep 17 14:21:25.150308 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:25.150266    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb2a1136-87dc-4692-9e19-cb692f34948a-credential-socket\") pod \"eb2a1136-87dc-4692-9e19-cb692f34948a\" (UID: \"eb2a1136-87dc-4692-9e19-cb692f34948a\") "
Sep 17 14:21:25.150404 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:25.150311    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb2a1136-87dc-4692-9e19-cb692f34948a-istio-envoy\") pod \"eb2a1136-87dc-4692-9e19-cb692f34948a\" (UID: \"eb2a1136-87dc-4692-9e19-cb692f34948a\") "
Sep 17 14:21:25.150468 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:25.150424    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb2a1136-87dc-4692-9e19-cb692f34948a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "eb2a1136-87dc-4692-9e19-cb692f34948a" (UID: "eb2a1136-87dc-4692-9e19-cb692f34948a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:21:25.150617 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:25.150577    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb2a1136-87dc-4692-9e19-cb692f34948a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "eb2a1136-87dc-4692-9e19-cb692f34948a" (UID: "eb2a1136-87dc-4692-9e19-cb692f34948a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:21:25.150834 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:25.150790    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb2a1136-87dc-4692-9e19-cb692f34948a-workload-socket\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:21:25.150834 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:25.150813    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb2a1136-87dc-4692-9e19-cb692f34948a-istiod-ca-cert\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:21:25.150834 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:25.150822    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb2a1136-87dc-4692-9e19-cb692f34948a-workload-certs\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:21:25.150834 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:25.150832    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb2a1136-87dc-4692-9e19-cb692f34948a-credential-socket\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:21:25.151085 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:25.150847    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb2a1136-87dc-4692-9e19-cb692f34948a-istio-data\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:21:25.152651 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:25.152585    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eb2a1136-87dc-4692-9e19-cb692f34948a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "eb2a1136-87dc-4692-9e19-cb692f34948a" (UID: "eb2a1136-87dc-4692-9e19-cb692f34948a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 14:21:25.152773 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:25.152742    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb2a1136-87dc-4692-9e19-cb692f34948a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "eb2a1136-87dc-4692-9e19-cb692f34948a" (UID: "eb2a1136-87dc-4692-9e19-cb692f34948a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:21:25.152843 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:25.152762    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb2a1136-87dc-4692-9e19-cb692f34948a-kube-api-access-4qld6" (OuterVolumeSpecName: "kube-api-access-4qld6") pod "eb2a1136-87dc-4692-9e19-cb692f34948a" (UID: "eb2a1136-87dc-4692-9e19-cb692f34948a"). InnerVolumeSpecName "kube-api-access-4qld6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:21:25.152891 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:25.152838    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb2a1136-87dc-4692-9e19-cb692f34948a-istio-token" (OuterVolumeSpecName: "istio-token") pod "eb2a1136-87dc-4692-9e19-cb692f34948a" (UID: "eb2a1136-87dc-4692-9e19-cb692f34948a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:21:25.251949 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:25.251912    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qld6\" (UniqueName: \"kubernetes.io/projected/eb2a1136-87dc-4692-9e19-cb692f34948a-kube-api-access-4qld6\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:21:25.251949 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:25.251950    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb2a1136-87dc-4692-9e19-cb692f34948a-istio-podinfo\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:21:25.252154 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:25.251963    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb2a1136-87dc-4692-9e19-cb692f34948a-istio-token\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:21:25.252154 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:25.251971    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb2a1136-87dc-4692-9e19-cb692f34948a-istio-envoy\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:21:25.841850 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:25.841759    2582 generic.go:356] "Generic (PLEG): container finished" podID="eb2a1136-87dc-4692-9e19-cb692f34948a" containerID="90f301fdb26acafb7fc06cdd48329a3f9468d2bf76178b4c11f1c835e4502a20" exitCode=0
Sep 17 14:21:25.841850 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:25.841810    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj"
Sep 17 14:21:25.842034 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:25.841855    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj" event={"ID":"eb2a1136-87dc-4692-9e19-cb692f34948a","Type":"ContainerDied","Data":"90f301fdb26acafb7fc06cdd48329a3f9468d2bf76178b4c11f1c835e4502a20"}
Sep 17 14:21:25.842034 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:25.841915    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj" event={"ID":"eb2a1136-87dc-4692-9e19-cb692f34948a","Type":"ContainerDied","Data":"ab4f57607e95a8b4727be06942654f101aa6d5a365b4bc9173554a3f561eaa42"}
Sep 17 14:21:25.842034 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:25.841940    2582 scope.go:117] "RemoveContainer" containerID="90f301fdb26acafb7fc06cdd48329a3f9468d2bf76178b4c11f1c835e4502a20"
Sep 17 14:21:25.851385 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:25.851368    2582 scope.go:117] "RemoveContainer" containerID="a9ab067c8a55aad076f883ba6b12c0039c549a6e8ff75980e8b7e72a4f97e689"
Sep 17 14:21:25.860353 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:25.860329    2582 scope.go:117] "RemoveContainer" containerID="90f301fdb26acafb7fc06cdd48329a3f9468d2bf76178b4c11f1c835e4502a20"
Sep 17 14:21:25.860668 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:21:25.860648    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90f301fdb26acafb7fc06cdd48329a3f9468d2bf76178b4c11f1c835e4502a20\": container with ID starting with 90f301fdb26acafb7fc06cdd48329a3f9468d2bf76178b4c11f1c835e4502a20 not found: ID does not exist" containerID="90f301fdb26acafb7fc06cdd48329a3f9468d2bf76178b4c11f1c835e4502a20"
Sep 17 14:21:25.860750 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:25.860676    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90f301fdb26acafb7fc06cdd48329a3f9468d2bf76178b4c11f1c835e4502a20"} err="failed to get container status \"90f301fdb26acafb7fc06cdd48329a3f9468d2bf76178b4c11f1c835e4502a20\": rpc error: code = NotFound desc = could not find container \"90f301fdb26acafb7fc06cdd48329a3f9468d2bf76178b4c11f1c835e4502a20\": container with ID starting with 90f301fdb26acafb7fc06cdd48329a3f9468d2bf76178b4c11f1c835e4502a20 not found: ID does not exist"
Sep 17 14:21:25.860750 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:25.860694    2582 scope.go:117] "RemoveContainer" containerID="a9ab067c8a55aad076f883ba6b12c0039c549a6e8ff75980e8b7e72a4f97e689"
Sep 17 14:21:25.860873 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:25.860854    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj"]
Sep 17 14:21:25.860962 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:21:25.860941    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9ab067c8a55aad076f883ba6b12c0039c549a6e8ff75980e8b7e72a4f97e689\": container with ID starting with a9ab067c8a55aad076f883ba6b12c0039c549a6e8ff75980e8b7e72a4f97e689 not found: ID does not exist" containerID="a9ab067c8a55aad076f883ba6b12c0039c549a6e8ff75980e8b7e72a4f97e689"
Sep 17 14:21:25.861001 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:25.860971    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9ab067c8a55aad076f883ba6b12c0039c549a6e8ff75980e8b7e72a4f97e689"} err="failed to get container status \"a9ab067c8a55aad076f883ba6b12c0039c549a6e8ff75980e8b7e72a4f97e689\": rpc error: code = NotFound desc = could not find container \"a9ab067c8a55aad076f883ba6b12c0039c549a6e8ff75980e8b7e72a4f97e689\": container with ID starting with a9ab067c8a55aad076f883ba6b12c0039c549a6e8ff75980e8b7e72a4f97e689 not found: ID does not exist"
Sep 17 14:21:25.865256 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:25.865231    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-99500ae5-openshift-default-6467c844b4-dtlrj"]
Sep 17 14:21:26.881210 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:26.881164    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:21:26.987860 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:26.987827    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:21:27.083020 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:27.082991    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:21:27.143331 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:27.143234    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:21:27.501949 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:27.501915    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb2a1136-87dc-4692-9e19-cb692f34948a" path="/var/lib/kubelet/pods/eb2a1136-87dc-4692-9e19-cb692f34948a/volumes"
Sep 17 14:21:27.598157 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:27.598119    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:21:46.752934 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:46.752897    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:21:47.976999 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:47.976965    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:21:48.918903 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:48.918871    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:21:49.038773 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:49.038733    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:21:53.537419 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:53.537379    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:21:59.292454 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:59.292420    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:21:59.397785 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:59.397751    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:21:59.493882 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:21:59.493852    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:01.606467 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:01.606435    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:01.725468 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:01.725432    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:01.888139 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:01.888041    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm"]
Sep 17 14:22:01.888359 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:01.888328    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm" podUID="7f56f2e1-a1f7-4512-8024-7ab51ac41a9c" containerName="istio-proxy" containerID="cri-o://e4d988bd77a3919dd99840bf7ebe7d1685a5b85f8732061369386a8a1f7a8e94" gracePeriod=30
Sep 17 14:22:01.901538 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:01.901511    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:02.346728 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:02.346681    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:02.626710 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:02.626598    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf"]
Sep 17 14:22:02.627239 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:02.627215    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2329a73f-b523-4312-a5d2-4fd886fcd46a" containerName="revoke-keys"
Sep 17 14:22:02.627239 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:02.627238    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2329a73f-b523-4312-a5d2-4fd886fcd46a" containerName="revoke-keys"
Sep 17 14:22:02.627441 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:02.627265    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb2a1136-87dc-4692-9e19-cb692f34948a" containerName="istio-proxy"
Sep 17 14:22:02.627441 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:02.627274    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb2a1136-87dc-4692-9e19-cb692f34948a" containerName="istio-proxy"
Sep 17 14:22:02.627441 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:02.627418    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb2a1136-87dc-4692-9e19-cb692f34948a" containerName="istio-proxy"
Sep 17 14:22:02.627441 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:02.627439    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2329a73f-b523-4312-a5d2-4fd886fcd46a" containerName="revoke-keys"
Sep 17 14:22:02.630572 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:02.630549    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf"
Sep 17 14:22:02.632848 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:02.632824    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-68183c-openshift-default-dockercfg-h6fwc\""
Sep 17 14:22:02.641727 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:02.641697    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf"]
Sep 17 14:22:02.653253 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:02.653228    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:02.747632 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:02.747569    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4802f1d6-92af-4711-ae3e-e85d0442a1ea-istio-data\") pod \"e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf\" (UID: \"4802f1d6-92af-4711-ae3e-e85d0442a1ea\") " pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf"
Sep 17 14:22:02.747834 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:02.747663    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4802f1d6-92af-4711-ae3e-e85d0442a1ea-istiod-ca-cert\") pod \"e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf\" (UID: \"4802f1d6-92af-4711-ae3e-e85d0442a1ea\") " pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf"
Sep 17 14:22:02.747834 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:02.747743    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4802f1d6-92af-4711-ae3e-e85d0442a1ea-istio-envoy\") pod \"e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf\" (UID: \"4802f1d6-92af-4711-ae3e-e85d0442a1ea\") " pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf"
Sep 17 14:22:02.747834 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:02.747814    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4802f1d6-92af-4711-ae3e-e85d0442a1ea-credential-socket\") pod \"e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf\" (UID: \"4802f1d6-92af-4711-ae3e-e85d0442a1ea\") " pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf"
Sep 17 14:22:02.748009 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:02.747872    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4802f1d6-92af-4711-ae3e-e85d0442a1ea-istio-podinfo\") pod \"e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf\" (UID: \"4802f1d6-92af-4711-ae3e-e85d0442a1ea\") " pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf"
Sep 17 14:22:02.748009 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:02.747917    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4802f1d6-92af-4711-ae3e-e85d0442a1ea-istio-token\") pod \"e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf\" (UID: \"4802f1d6-92af-4711-ae3e-e85d0442a1ea\") " pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf"
Sep 17 14:22:02.748009 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:02.747990    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpvtv\" (UniqueName: \"kubernetes.io/projected/4802f1d6-92af-4711-ae3e-e85d0442a1ea-kube-api-access-lpvtv\") pod \"e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf\" (UID: \"4802f1d6-92af-4711-ae3e-e85d0442a1ea\") " pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf"
Sep 17 14:22:02.748165 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:02.748047    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4802f1d6-92af-4711-ae3e-e85d0442a1ea-workload-socket\") pod \"e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf\" (UID: \"4802f1d6-92af-4711-ae3e-e85d0442a1ea\") " pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf"
Sep 17 14:22:02.748165 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:02.748087    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4802f1d6-92af-4711-ae3e-e85d0442a1ea-workload-certs\") pod \"e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf\" (UID: \"4802f1d6-92af-4711-ae3e-e85d0442a1ea\") " pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf"
Sep 17 14:22:02.849732 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:02.849675    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4802f1d6-92af-4711-ae3e-e85d0442a1ea-istio-token\") pod \"e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf\" (UID: \"4802f1d6-92af-4711-ae3e-e85d0442a1ea\") " pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf"
Sep 17 14:22:02.849945 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:02.849762    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpvtv\" (UniqueName: \"kubernetes.io/projected/4802f1d6-92af-4711-ae3e-e85d0442a1ea-kube-api-access-lpvtv\") pod \"e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf\" (UID: \"4802f1d6-92af-4711-ae3e-e85d0442a1ea\") " pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf"
Sep 17 14:22:02.849945 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:02.849807    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4802f1d6-92af-4711-ae3e-e85d0442a1ea-workload-socket\") pod \"e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf\" (UID: \"4802f1d6-92af-4711-ae3e-e85d0442a1ea\") " pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf"
Sep 17 14:22:02.849945 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:02.849839    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4802f1d6-92af-4711-ae3e-e85d0442a1ea-workload-certs\") pod \"e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf\" (UID: \"4802f1d6-92af-4711-ae3e-e85d0442a1ea\") " pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf"
Sep 17 14:22:02.849945 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:02.849873    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4802f1d6-92af-4711-ae3e-e85d0442a1ea-istio-data\") pod \"e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf\" (UID: \"4802f1d6-92af-4711-ae3e-e85d0442a1ea\") " pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf"
Sep 17 14:22:02.849945 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:02.849901    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4802f1d6-92af-4711-ae3e-e85d0442a1ea-istiod-ca-cert\") pod \"e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf\" (UID: \"4802f1d6-92af-4711-ae3e-e85d0442a1ea\") " pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf"
Sep 17 14:22:02.850211 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:02.849960    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4802f1d6-92af-4711-ae3e-e85d0442a1ea-istio-envoy\") pod \"e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf\" (UID: \"4802f1d6-92af-4711-ae3e-e85d0442a1ea\") " pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf"
Sep 17 14:22:02.850211 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:02.850015    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4802f1d6-92af-4711-ae3e-e85d0442a1ea-credential-socket\") pod \"e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf\" (UID: \"4802f1d6-92af-4711-ae3e-e85d0442a1ea\") " pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf"
Sep 17 14:22:02.850211 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:02.850057    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4802f1d6-92af-4711-ae3e-e85d0442a1ea-istio-podinfo\") pod \"e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf\" (UID: \"4802f1d6-92af-4711-ae3e-e85d0442a1ea\") " pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf"
Sep 17 14:22:02.851092 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:02.851063    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4802f1d6-92af-4711-ae3e-e85d0442a1ea-workload-socket\") pod \"e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf\" (UID: \"4802f1d6-92af-4711-ae3e-e85d0442a1ea\") " pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf"
Sep 17 14:22:02.851284 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:02.851110    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4802f1d6-92af-4711-ae3e-e85d0442a1ea-istio-data\") pod \"e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf\" (UID: \"4802f1d6-92af-4711-ae3e-e85d0442a1ea\") " pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf"
Sep 17 14:22:02.851402 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:02.851323    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4802f1d6-92af-4711-ae3e-e85d0442a1ea-istiod-ca-cert\") pod \"e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf\" (UID: \"4802f1d6-92af-4711-ae3e-e85d0442a1ea\") " pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf"
Sep 17 14:22:02.851621 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:02.851582    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4802f1d6-92af-4711-ae3e-e85d0442a1ea-credential-socket\") pod \"e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf\" (UID: \"4802f1d6-92af-4711-ae3e-e85d0442a1ea\") " pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf"
Sep 17 14:22:02.851798 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:02.851729    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4802f1d6-92af-4711-ae3e-e85d0442a1ea-workload-certs\") pod \"e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf\" (UID: \"4802f1d6-92af-4711-ae3e-e85d0442a1ea\") " pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf"
Sep 17 14:22:02.853775 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:02.853754    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4802f1d6-92af-4711-ae3e-e85d0442a1ea-istio-envoy\") pod \"e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf\" (UID: \"4802f1d6-92af-4711-ae3e-e85d0442a1ea\") " pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf"
Sep 17 14:22:02.853879 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:02.853815    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4802f1d6-92af-4711-ae3e-e85d0442a1ea-istio-podinfo\") pod \"e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf\" (UID: \"4802f1d6-92af-4711-ae3e-e85d0442a1ea\") " pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf"
Sep 17 14:22:02.858100 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:02.858073    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpvtv\" (UniqueName: \"kubernetes.io/projected/4802f1d6-92af-4711-ae3e-e85d0442a1ea-kube-api-access-lpvtv\") pod \"e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf\" (UID: \"4802f1d6-92af-4711-ae3e-e85d0442a1ea\") " pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf"
Sep 17 14:22:02.858100 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:02.858093    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4802f1d6-92af-4711-ae3e-e85d0442a1ea-istio-token\") pod \"e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf\" (UID: \"4802f1d6-92af-4711-ae3e-e85d0442a1ea\") " pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf"
Sep 17 14:22:02.897014 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:02.896941    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:02.943737 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:02.943696    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf"
Sep 17 14:22:03.221447 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:03.221423    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf"]
Sep 17 14:22:03.224448 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:22:03.224416    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4802f1d6_92af_4711_ae3e_e85d0442a1ea.slice/crio-4c9976f2c1f268399f98a60d03dbb1a523a15d6a746e00bbc3e317c95ef80ef7 WatchSource:0}: Error finding container 4c9976f2c1f268399f98a60d03dbb1a523a15d6a746e00bbc3e317c95ef80ef7: Status 404 returned error can't find the container with id 4c9976f2c1f268399f98a60d03dbb1a523a15d6a746e00bbc3e317c95ef80ef7
Sep 17 14:22:03.226900 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:03.226857    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:22:03.226995 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:03.226951    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:22:03.227043 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:03.226994    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:22:04.068810 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:04.068771    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf" event={"ID":"4802f1d6-92af-4711-ae3e-e85d0442a1ea","Type":"ContainerStarted","Data":"c3cd5ebd47ac07b15b2a05767a0dfb0dacdd6f48b9e838b206bbeb6248eb0fcb"}
Sep 17 14:22:04.069229 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:04.068833    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf" event={"ID":"4802f1d6-92af-4711-ae3e-e85d0442a1ea","Type":"ContainerStarted","Data":"4c9976f2c1f268399f98a60d03dbb1a523a15d6a746e00bbc3e317c95ef80ef7"}
Sep 17 14:22:04.089859 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:04.089813    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf" podStartSLOduration=2.089796133 podStartE2EDuration="2.089796133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:22:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:22:04.088474548 +0000 UTC m=+1025.090018023" watchObservedRunningTime="2026-09-17 14:22:04.089796133 +0000 UTC m=+1025.091339606"
Sep 17 14:22:04.945000 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:04.944965    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf"
Sep 17 14:22:04.950778 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:04.950753    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf"
Sep 17 14:22:05.075512 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:05.075471    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf"
Sep 17 14:22:05.076497 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:05.076478    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf"
Sep 17 14:22:05.315521 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:05.315436    2582 patch_prober.go:28] interesting pod/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 17 14:22:05.315521 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:05.315502    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm" podUID="7f56f2e1-a1f7-4512-8024-7ab51ac41a9c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 14:22:05.914874 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:05.914838    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:06.017202 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:06.017137    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:06.118501 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:06.118465    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:06.736114 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:06.736082    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:07.093268 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:07.093236    2582 generic.go:356] "Generic (PLEG): container finished" podID="7f56f2e1-a1f7-4512-8024-7ab51ac41a9c" containerID="e4d988bd77a3919dd99840bf7ebe7d1685a5b85f8732061369386a8a1f7a8e94" exitCode=0
Sep 17 14:22:07.093463 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:07.093311    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm" event={"ID":"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c","Type":"ContainerDied","Data":"e4d988bd77a3919dd99840bf7ebe7d1685a5b85f8732061369386a8a1f7a8e94"}
Sep 17 14:22:07.093463 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:07.093369    2582 scope.go:117] "RemoveContainer" containerID="cbd65e276f0944ef708a40b87442b53da7c76bebb218ac24d0c9bc1306177057"
Sep 17 14:22:07.151483 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:07.151453    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm"
Sep 17 14:22:07.193050 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:07.191984    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-workload-certs\") pod \"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c\" (UID: \"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c\") "
Sep 17 14:22:07.193050 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:07.192063    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-istio-data\") pod \"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c\" (UID: \"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c\") "
Sep 17 14:22:07.193050 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:07.192117    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-istio-token\") pod \"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c\" (UID: \"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c\") "
Sep 17 14:22:07.193050 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:07.192145    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-istiod-ca-cert\") pod \"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c\" (UID: \"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c\") "
Sep 17 14:22:07.193050 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:07.192197    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wkxbk\" (UniqueName: \"kubernetes.io/projected/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-kube-api-access-wkxbk\") pod \"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c\" (UID: \"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c\") "
Sep 17 14:22:07.193050 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:07.192237    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-credential-socket\") pod \"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c\" (UID: \"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c\") "
Sep 17 14:22:07.193050 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:07.192261    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-istio-podinfo\") pod \"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c\" (UID: \"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c\") "
Sep 17 14:22:07.193050 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:07.192305    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-workload-socket\") pod \"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c\" (UID: \"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c\") "
Sep 17 14:22:07.193050 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:07.192368    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-istio-envoy\") pod \"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c\" (UID: \"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c\") "
Sep 17 14:22:07.193653 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:07.193236    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7f56f2e1-a1f7-4512-8024-7ab51ac41a9c" (UID: "7f56f2e1-a1f7-4512-8024-7ab51ac41a9c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:22:07.193653 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:07.193443    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-istio-data" (OuterVolumeSpecName: "istio-data") pod "7f56f2e1-a1f7-4512-8024-7ab51ac41a9c" (UID: "7f56f2e1-a1f7-4512-8024-7ab51ac41a9c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:22:07.194911 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:07.194529    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7f56f2e1-a1f7-4512-8024-7ab51ac41a9c" (UID: "7f56f2e1-a1f7-4512-8024-7ab51ac41a9c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:22:07.197848 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:07.197558    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-istio-token" (OuterVolumeSpecName: "istio-token") pod "7f56f2e1-a1f7-4512-8024-7ab51ac41a9c" (UID: "7f56f2e1-a1f7-4512-8024-7ab51ac41a9c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:22:07.204596 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:07.204573    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7f56f2e1-a1f7-4512-8024-7ab51ac41a9c" (UID: "7f56f2e1-a1f7-4512-8024-7ab51ac41a9c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:22:07.204949 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:07.204928    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7f56f2e1-a1f7-4512-8024-7ab51ac41a9c" (UID: "7f56f2e1-a1f7-4512-8024-7ab51ac41a9c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:22:07.207311 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:07.207259    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7f56f2e1-a1f7-4512-8024-7ab51ac41a9c" (UID: "7f56f2e1-a1f7-4512-8024-7ab51ac41a9c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 14:22:07.207672 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:07.207577    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-kube-api-access-wkxbk" (OuterVolumeSpecName: "kube-api-access-wkxbk") pod "7f56f2e1-a1f7-4512-8024-7ab51ac41a9c" (UID: "7f56f2e1-a1f7-4512-8024-7ab51ac41a9c"). InnerVolumeSpecName "kube-api-access-wkxbk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:22:07.208905 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:07.208880    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7f56f2e1-a1f7-4512-8024-7ab51ac41a9c" (UID: "7f56f2e1-a1f7-4512-8024-7ab51ac41a9c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:22:07.297684 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:07.296201    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wkxbk\" (UniqueName: \"kubernetes.io/projected/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-kube-api-access-wkxbk\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:22:07.297684 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:07.296243    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-credential-socket\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:22:07.297684 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:07.296258    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-istio-podinfo\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:22:07.297684 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:07.296270    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-workload-socket\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:22:07.297684 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:07.296287    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-istio-envoy\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:22:07.297684 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:07.296311    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-workload-certs\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:22:07.297684 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:07.296322    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-istio-data\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:22:07.297684 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:07.296334    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-istio-token\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:22:07.297684 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:07.296350    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c-istiod-ca-cert\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:22:07.719502 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:07.719469    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:08.100675 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:08.100562    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm"
Sep 17 14:22:08.100675 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:08.100589    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm" event={"ID":"7f56f2e1-a1f7-4512-8024-7ab51ac41a9c","Type":"ContainerDied","Data":"c18892ff6e8b79409ff8b49068bceaad60248be9d8c467ff045ca9efe5095b69"}
Sep 17 14:22:08.100675 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:08.100675    2582 scope.go:117] "RemoveContainer" containerID="e4d988bd77a3919dd99840bf7ebe7d1685a5b85f8732061369386a8a1f7a8e94"
Sep 17 14:22:08.120574 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:08.120542    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm"]
Sep 17 14:22:08.124761 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:08.124734    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-3f6f32-openshift-default-6dd4df7db-q9npm"]
Sep 17 14:22:08.504712 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:08.504677    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:09.162851 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:09.162820    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:09.264081 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:09.264048    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:09.449534 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:09.449499    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:09.494112 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:09.494081    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f56f2e1-a1f7-4512-8024-7ab51ac41a9c" path="/var/lib/kubelet/pods/7f56f2e1-a1f7-4512-8024-7ab51ac41a9c/volumes"
Sep 17 14:22:09.629236 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:09.629197    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:09.728036 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:09.727936    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:15.361840 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:15.361801    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:18.180621 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:18.180584    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:21.409570 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:21.409534    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:21.510998 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:21.510971    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:21.626938 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:21.626900    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:31.001570 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:31.001521    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:31.108885 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:31.108850    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:31.245593 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:31.245557    2582 generic.go:356] "Generic (PLEG): container finished" podID="4802f1d6-92af-4711-ae3e-e85d0442a1ea" containerID="c3cd5ebd47ac07b15b2a05767a0dfb0dacdd6f48b9e838b206bbeb6248eb0fcb" exitCode=0
Sep 17 14:22:31.245803 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:31.245636    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf" event={"ID":"4802f1d6-92af-4711-ae3e-e85d0442a1ea","Type":"ContainerDied","Data":"c3cd5ebd47ac07b15b2a05767a0dfb0dacdd6f48b9e838b206bbeb6248eb0fcb"}
Sep 17 14:22:31.246036 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:31.246022    2582 scope.go:117] "RemoveContainer" containerID="c3cd5ebd47ac07b15b2a05767a0dfb0dacdd6f48b9e838b206bbeb6248eb0fcb"
Sep 17 14:22:31.247106 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:31.247074    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:22:31.247211 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:31.247163    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:22:31.247274 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:31.247212    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:22:31.824258 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:31.824227    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:32.040060 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.040010    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-567467b958-q9h5x"]
Sep 17 14:22:32.040949 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.040907    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f56f2e1-a1f7-4512-8024-7ab51ac41a9c" containerName="istio-proxy"
Sep 17 14:22:32.040949 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.040949    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f56f2e1-a1f7-4512-8024-7ab51ac41a9c" containerName="istio-proxy"
Sep 17 14:22:32.041122 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.040968    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb2a1136-87dc-4692-9e19-cb692f34948a" containerName="istio-proxy"
Sep 17 14:22:32.041122 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.040978    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb2a1136-87dc-4692-9e19-cb692f34948a" containerName="istio-proxy"
Sep 17 14:22:32.041122 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.041022    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f56f2e1-a1f7-4512-8024-7ab51ac41a9c" containerName="istio-proxy"
Sep 17 14:22:32.041122 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.041031    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f56f2e1-a1f7-4512-8024-7ab51ac41a9c" containerName="istio-proxy"
Sep 17 14:22:32.041325 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.041185    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f56f2e1-a1f7-4512-8024-7ab51ac41a9c" containerName="istio-proxy"
Sep 17 14:22:32.041325 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.041200    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f56f2e1-a1f7-4512-8024-7ab51ac41a9c" containerName="istio-proxy"
Sep 17 14:22:32.041325 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.041216    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb2a1136-87dc-4692-9e19-cb692f34948a" containerName="istio-proxy"
Sep 17 14:22:32.044104 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.044073    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-567467b958-q9h5x"
Sep 17 14:22:32.060435 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.060389    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-567467b958-q9h5x"]
Sep 17 14:22:32.107336 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.107243    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:32.163916 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.163878    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3c4bd392-74a7-41ce-acf8-de0ff81c4a44-oauth-serving-cert\") pod \"console-567467b958-q9h5x\" (UID: \"3c4bd392-74a7-41ce-acf8-de0ff81c4a44\") " pod="openshift-console/console-567467b958-q9h5x"
Sep 17 14:22:32.164093 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.163939    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3c4bd392-74a7-41ce-acf8-de0ff81c4a44-console-oauth-config\") pod \"console-567467b958-q9h5x\" (UID: \"3c4bd392-74a7-41ce-acf8-de0ff81c4a44\") " pod="openshift-console/console-567467b958-q9h5x"
Sep 17 14:22:32.164093 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.163963    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3c4bd392-74a7-41ce-acf8-de0ff81c4a44-console-config\") pod \"console-567467b958-q9h5x\" (UID: \"3c4bd392-74a7-41ce-acf8-de0ff81c4a44\") " pod="openshift-console/console-567467b958-q9h5x"
Sep 17 14:22:32.164093 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.163996    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c4bd392-74a7-41ce-acf8-de0ff81c4a44-console-serving-cert\") pod \"console-567467b958-q9h5x\" (UID: \"3c4bd392-74a7-41ce-acf8-de0ff81c4a44\") " pod="openshift-console/console-567467b958-q9h5x"
Sep 17 14:22:32.164093 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.164062    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2qhc\" (UniqueName: \"kubernetes.io/projected/3c4bd392-74a7-41ce-acf8-de0ff81c4a44-kube-api-access-x2qhc\") pod \"console-567467b958-q9h5x\" (UID: \"3c4bd392-74a7-41ce-acf8-de0ff81c4a44\") " pod="openshift-console/console-567467b958-q9h5x"
Sep 17 14:22:32.164232 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.164122    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c4bd392-74a7-41ce-acf8-de0ff81c4a44-trusted-ca-bundle\") pod \"console-567467b958-q9h5x\" (UID: \"3c4bd392-74a7-41ce-acf8-de0ff81c4a44\") " pod="openshift-console/console-567467b958-q9h5x"
Sep 17 14:22:32.164232 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.164142    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3c4bd392-74a7-41ce-acf8-de0ff81c4a44-service-ca\") pod \"console-567467b958-q9h5x\" (UID: \"3c4bd392-74a7-41ce-acf8-de0ff81c4a44\") " pod="openshift-console/console-567467b958-q9h5x"
Sep 17 14:22:32.205349 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.205312    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:32.257930 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.257888    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf" event={"ID":"4802f1d6-92af-4711-ae3e-e85d0442a1ea","Type":"ContainerStarted","Data":"83fb30129881040c86f166e14c797dab7d1dcec817d17c57a21e050d51b5cefb"}
Sep 17 14:22:32.264962 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.264940    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3c4bd392-74a7-41ce-acf8-de0ff81c4a44-console-oauth-config\") pod \"console-567467b958-q9h5x\" (UID: \"3c4bd392-74a7-41ce-acf8-de0ff81c4a44\") " pod="openshift-console/console-567467b958-q9h5x"
Sep 17 14:22:32.265135 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.264966    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3c4bd392-74a7-41ce-acf8-de0ff81c4a44-console-config\") pod \"console-567467b958-q9h5x\" (UID: \"3c4bd392-74a7-41ce-acf8-de0ff81c4a44\") " pod="openshift-console/console-567467b958-q9h5x"
Sep 17 14:22:32.265135 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.264994    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c4bd392-74a7-41ce-acf8-de0ff81c4a44-console-serving-cert\") pod \"console-567467b958-q9h5x\" (UID: \"3c4bd392-74a7-41ce-acf8-de0ff81c4a44\") " pod="openshift-console/console-567467b958-q9h5x"
Sep 17 14:22:32.265135 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.265020    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2qhc\" (UniqueName: \"kubernetes.io/projected/3c4bd392-74a7-41ce-acf8-de0ff81c4a44-kube-api-access-x2qhc\") pod \"console-567467b958-q9h5x\" (UID: \"3c4bd392-74a7-41ce-acf8-de0ff81c4a44\") " pod="openshift-console/console-567467b958-q9h5x"
Sep 17 14:22:32.265320 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.265154    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c4bd392-74a7-41ce-acf8-de0ff81c4a44-trusted-ca-bundle\") pod \"console-567467b958-q9h5x\" (UID: \"3c4bd392-74a7-41ce-acf8-de0ff81c4a44\") " pod="openshift-console/console-567467b958-q9h5x"
Sep 17 14:22:32.265320 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.265202    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3c4bd392-74a7-41ce-acf8-de0ff81c4a44-service-ca\") pod \"console-567467b958-q9h5x\" (UID: \"3c4bd392-74a7-41ce-acf8-de0ff81c4a44\") " pod="openshift-console/console-567467b958-q9h5x"
Sep 17 14:22:32.265320 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.265276    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3c4bd392-74a7-41ce-acf8-de0ff81c4a44-oauth-serving-cert\") pod \"console-567467b958-q9h5x\" (UID: \"3c4bd392-74a7-41ce-acf8-de0ff81c4a44\") " pod="openshift-console/console-567467b958-q9h5x"
Sep 17 14:22:32.265773 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.265746    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3c4bd392-74a7-41ce-acf8-de0ff81c4a44-console-config\") pod \"console-567467b958-q9h5x\" (UID: \"3c4bd392-74a7-41ce-acf8-de0ff81c4a44\") " pod="openshift-console/console-567467b958-q9h5x"
Sep 17 14:22:32.265963 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.265945    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3c4bd392-74a7-41ce-acf8-de0ff81c4a44-oauth-serving-cert\") pod \"console-567467b958-q9h5x\" (UID: \"3c4bd392-74a7-41ce-acf8-de0ff81c4a44\") " pod="openshift-console/console-567467b958-q9h5x"
Sep 17 14:22:32.266049 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.266031    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c4bd392-74a7-41ce-acf8-de0ff81c4a44-trusted-ca-bundle\") pod \"console-567467b958-q9h5x\" (UID: \"3c4bd392-74a7-41ce-acf8-de0ff81c4a44\") " pod="openshift-console/console-567467b958-q9h5x"
Sep 17 14:22:32.266119 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.266079    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3c4bd392-74a7-41ce-acf8-de0ff81c4a44-service-ca\") pod \"console-567467b958-q9h5x\" (UID: \"3c4bd392-74a7-41ce-acf8-de0ff81c4a44\") " pod="openshift-console/console-567467b958-q9h5x"
Sep 17 14:22:32.267539 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.267510    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3c4bd392-74a7-41ce-acf8-de0ff81c4a44-console-oauth-config\") pod \"console-567467b958-q9h5x\" (UID: \"3c4bd392-74a7-41ce-acf8-de0ff81c4a44\") " pod="openshift-console/console-567467b958-q9h5x"
Sep 17 14:22:32.267727 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.267710    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c4bd392-74a7-41ce-acf8-de0ff81c4a44-console-serving-cert\") pod \"console-567467b958-q9h5x\" (UID: \"3c4bd392-74a7-41ce-acf8-de0ff81c4a44\") " pod="openshift-console/console-567467b958-q9h5x"
Sep 17 14:22:32.274952 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.274931    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2qhc\" (UniqueName: \"kubernetes.io/projected/3c4bd392-74a7-41ce-acf8-de0ff81c4a44-kube-api-access-x2qhc\") pod \"console-567467b958-q9h5x\" (UID: \"3c4bd392-74a7-41ce-acf8-de0ff81c4a44\") " pod="openshift-console/console-567467b958-q9h5x"
Sep 17 14:22:32.346345 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.346297    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:32.366182 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.366079    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-567467b958-q9h5x"
Sep 17 14:22:32.469279 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.466198    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:32.637335 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.637064    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:32.670052 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.670024    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-567467b958-q9h5x"]
Sep 17 14:22:32.671675 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:22:32.671640    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c4bd392_74a7_41ce_acf8_de0ff81c4a44.slice/crio-9a067657c29f29fed998c547aa587908700b6515b3986080083e53a0c9fe1ff8 WatchSource:0}: Error finding container 9a067657c29f29fed998c547aa587908700b6515b3986080083e53a0c9fe1ff8: Status 404 returned error can't find the container with id 9a067657c29f29fed998c547aa587908700b6515b3986080083e53a0c9fe1ff8
Sep 17 14:22:32.832069 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.831854    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:32.933821 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.933554    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:32.945043 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.944956    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf"
Sep 17 14:22:32.946385 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.946347    2582 patch_prober.go:28] interesting pod/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Sep 17 14:22:32.946540 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:32.946411    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf" podUID="4802f1d6-92af-4711-ae3e-e85d0442a1ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Sep 17 14:22:33.034094 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:33.033984    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:33.266299 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:33.266256    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-567467b958-q9h5x" event={"ID":"3c4bd392-74a7-41ce-acf8-de0ff81c4a44","Type":"ContainerStarted","Data":"ef10ef817df5dad1bee76ac2b1b32a165ece50ccb88543aa94e8a86d7dfc4d25"}
Sep 17 14:22:33.266823 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:33.266317    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-567467b958-q9h5x" event={"ID":"3c4bd392-74a7-41ce-acf8-de0ff81c4a44","Type":"ContainerStarted","Data":"9a067657c29f29fed998c547aa587908700b6515b3986080083e53a0c9fe1ff8"}
Sep 17 14:22:33.288931 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:33.288878    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-567467b958-q9h5x" podStartSLOduration=1.288860946 podStartE2EDuration="1.288860946s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:22:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:22:33.285927563 +0000 UTC m=+1054.287471038" watchObservedRunningTime="2026-09-17 14:22:33.288860946 +0000 UTC m=+1054.290404420"
Sep 17 14:22:33.605772 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:33.605692    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:33.945555 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:33.945516    2582 patch_prober.go:28] interesting pod/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Sep 17 14:22:33.945773 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:33.945592    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf" podUID="4802f1d6-92af-4711-ae3e-e85d0442a1ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Sep 17 14:22:34.232453 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:34.232300    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:34.944966 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:34.944924    2582 patch_prober.go:28] interesting pod/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Sep 17 14:22:34.945573 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:34.945534    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf" podUID="4802f1d6-92af-4711-ae3e-e85d0442a1ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Sep 17 14:22:35.040688 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:35.040648    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:35.076428 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:35.076391    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf"
Sep 17 14:22:35.945040 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:35.944990    2582 patch_prober.go:28] interesting pod/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Sep 17 14:22:35.945669 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:35.945061    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf" podUID="4802f1d6-92af-4711-ae3e-e85d0442a1ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Sep 17 14:22:36.441692 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:36.440665    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:36.945196 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:36.945147    2582 patch_prober.go:28] interesting pod/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Sep 17 14:22:36.945714 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:36.945245    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf" podUID="4802f1d6-92af-4711-ae3e-e85d0442a1ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Sep 17 14:22:37.948788 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:37.948751    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf"
Sep 17 14:22:37.949855 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:37.949838    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf"
Sep 17 14:22:38.067588 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:38.067543    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:38.168095 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:38.168060    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:39.122913 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:39.122880    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:39.225877 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:39.225838    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:39.325047 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:39.325014    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:39.976759 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:39.976712    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:40.274042 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:40.273958    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:41.662103 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:41.662066    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:42.366483 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:42.366448    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-567467b958-q9h5x"
Sep 17 14:22:42.366779 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:42.366758    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-567467b958-q9h5x"
Sep 17 14:22:42.371442 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:42.371419    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-567467b958-q9h5x"
Sep 17 14:22:43.336271 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:43.336242    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-567467b958-q9h5x"
Sep 17 14:22:43.396411 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:43.396377    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64585d4699-8csw4"]
Sep 17 14:22:43.497921 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:43.497871    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:43.599504 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:43.599418    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:43.699839 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:43.699800    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:47.989684 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:47.989648    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:48.095134 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:48.095102    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:48.670589 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:48.670556    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:49.389818 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:49.389786    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-9b8115/test-model-9b8115-kserve-7896bbcf7c-x294s"]
Sep 17 14:22:49.393120 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:49.393100    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-9b8115/test-model-9b8115-kserve-7896bbcf7c-x294s"
Sep 17 14:22:49.396598 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:49.396574    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-9b8115\"/\"openshift-service-ca.crt\""
Sep 17 14:22:49.396717 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:49.396584    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-9b8115\"/\"test-model-9b8115-kserve-self-signed-certs\""
Sep 17 14:22:49.396717 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:49.396664    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-9b8115\"/\"default-dockercfg-npvgn\""
Sep 17 14:22:49.396824 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:49.396587    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-9b8115\"/\"kube-root-ca.crt\""
Sep 17 14:22:49.406149 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:49.406125    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-9b8115/test-model-9b8115-kserve-7896bbcf7c-x294s"]
Sep 17 14:22:49.417860 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:49.417826    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:49.566031 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:49.565998    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb-home\") pod \"test-model-9b8115-kserve-7896bbcf7c-x294s\" (UID: \"1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb\") " pod="ai-tenant-e2e-inf-b-9b8115/test-model-9b8115-kserve-7896bbcf7c-x294s"
Sep 17 14:22:49.566031 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:49.566035    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb-tmp-dir\") pod \"test-model-9b8115-kserve-7896bbcf7c-x294s\" (UID: \"1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb\") " pod="ai-tenant-e2e-inf-b-9b8115/test-model-9b8115-kserve-7896bbcf7c-x294s"
Sep 17 14:22:49.566263 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:49.566121    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb-tls-certs\") pod \"test-model-9b8115-kserve-7896bbcf7c-x294s\" (UID: \"1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb\") " pod="ai-tenant-e2e-inf-b-9b8115/test-model-9b8115-kserve-7896bbcf7c-x294s"
Sep 17 14:22:49.566263 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:49.566182    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb-model-cache\") pod \"test-model-9b8115-kserve-7896bbcf7c-x294s\" (UID: \"1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb\") " pod="ai-tenant-e2e-inf-b-9b8115/test-model-9b8115-kserve-7896bbcf7c-x294s"
Sep 17 14:22:49.566263 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:49.566236    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thlmh\" (UniqueName: \"kubernetes.io/projected/1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb-kube-api-access-thlmh\") pod \"test-model-9b8115-kserve-7896bbcf7c-x294s\" (UID: \"1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb\") " pod="ai-tenant-e2e-inf-b-9b8115/test-model-9b8115-kserve-7896bbcf7c-x294s"
Sep 17 14:22:49.566363 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:49.566339    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb-dshm\") pod \"test-model-9b8115-kserve-7896bbcf7c-x294s\" (UID: \"1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb\") " pod="ai-tenant-e2e-inf-b-9b8115/test-model-9b8115-kserve-7896bbcf7c-x294s"
Sep 17 14:22:49.667550 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:49.667453    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb-tls-certs\") pod \"test-model-9b8115-kserve-7896bbcf7c-x294s\" (UID: \"1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb\") " pod="ai-tenant-e2e-inf-b-9b8115/test-model-9b8115-kserve-7896bbcf7c-x294s"
Sep 17 14:22:49.667550 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:49.667496    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb-model-cache\") pod \"test-model-9b8115-kserve-7896bbcf7c-x294s\" (UID: \"1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb\") " pod="ai-tenant-e2e-inf-b-9b8115/test-model-9b8115-kserve-7896bbcf7c-x294s"
Sep 17 14:22:49.667550 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:49.667534    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thlmh\" (UniqueName: \"kubernetes.io/projected/1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb-kube-api-access-thlmh\") pod \"test-model-9b8115-kserve-7896bbcf7c-x294s\" (UID: \"1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb\") " pod="ai-tenant-e2e-inf-b-9b8115/test-model-9b8115-kserve-7896bbcf7c-x294s"
Sep 17 14:22:49.667865 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:49.667622    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb-dshm\") pod \"test-model-9b8115-kserve-7896bbcf7c-x294s\" (UID: \"1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb\") " pod="ai-tenant-e2e-inf-b-9b8115/test-model-9b8115-kserve-7896bbcf7c-x294s"
Sep 17 14:22:49.667865 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:49.667650    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb-home\") pod \"test-model-9b8115-kserve-7896bbcf7c-x294s\" (UID: \"1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb\") " pod="ai-tenant-e2e-inf-b-9b8115/test-model-9b8115-kserve-7896bbcf7c-x294s"
Sep 17 14:22:49.667865 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:49.667667    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb-tmp-dir\") pod \"test-model-9b8115-kserve-7896bbcf7c-x294s\" (UID: \"1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb\") " pod="ai-tenant-e2e-inf-b-9b8115/test-model-9b8115-kserve-7896bbcf7c-x294s"
Sep 17 14:22:49.668031 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:49.667964    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb-model-cache\") pod \"test-model-9b8115-kserve-7896bbcf7c-x294s\" (UID: \"1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb\") " pod="ai-tenant-e2e-inf-b-9b8115/test-model-9b8115-kserve-7896bbcf7c-x294s"
Sep 17 14:22:49.668031 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:49.668008    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb-home\") pod \"test-model-9b8115-kserve-7896bbcf7c-x294s\" (UID: \"1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb\") " pod="ai-tenant-e2e-inf-b-9b8115/test-model-9b8115-kserve-7896bbcf7c-x294s"
Sep 17 14:22:49.668173 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:49.668153    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb-tmp-dir\") pod \"test-model-9b8115-kserve-7896bbcf7c-x294s\" (UID: \"1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb\") " pod="ai-tenant-e2e-inf-b-9b8115/test-model-9b8115-kserve-7896bbcf7c-x294s"
Sep 17 14:22:49.669958 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:49.669928    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb-dshm\") pod \"test-model-9b8115-kserve-7896bbcf7c-x294s\" (UID: \"1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb\") " pod="ai-tenant-e2e-inf-b-9b8115/test-model-9b8115-kserve-7896bbcf7c-x294s"
Sep 17 14:22:49.670060 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:49.670031    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb-tls-certs\") pod \"test-model-9b8115-kserve-7896bbcf7c-x294s\" (UID: \"1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb\") " pod="ai-tenant-e2e-inf-b-9b8115/test-model-9b8115-kserve-7896bbcf7c-x294s"
Sep 17 14:22:49.675741 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:49.675721    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thlmh\" (UniqueName: \"kubernetes.io/projected/1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb-kube-api-access-thlmh\") pod \"test-model-9b8115-kserve-7896bbcf7c-x294s\" (UID: \"1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb\") " pod="ai-tenant-e2e-inf-b-9b8115/test-model-9b8115-kserve-7896bbcf7c-x294s"
Sep 17 14:22:49.705573 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:49.705538    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-9b8115/test-model-9b8115-kserve-7896bbcf7c-x294s"
Sep 17 14:22:49.983377 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:49.983351    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-9b8115/test-model-9b8115-kserve-7896bbcf7c-x294s"]
Sep 17 14:22:49.985149 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:22:49.985118    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1d683c3f_6bfc_44a5_aa45_d9956a9ee5fb.slice/crio-f1a085435a66f59b90979792e7a477b47e1b34d0863189765b7f6583f0747522 WatchSource:0}: Error finding container f1a085435a66f59b90979792e7a477b47e1b34d0863189765b7f6583f0747522: Status 404 returned error can't find the container with id f1a085435a66f59b90979792e7a477b47e1b34d0863189765b7f6583f0747522
Sep 17 14:22:50.377929 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:50.377842    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-9b8115/test-model-9b8115-kserve-7896bbcf7c-x294s" event={"ID":"1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb","Type":"ContainerStarted","Data":"f1a085435a66f59b90979792e7a477b47e1b34d0863189765b7f6583f0747522"}
Sep 17 14:22:51.385419 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:51.385331    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-9b8115/test-model-9b8115-kserve-7896bbcf7c-x294s" event={"ID":"1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb","Type":"ContainerStarted","Data":"69e7e889313379cf09e7e02a8e9b78bc6ff88f1c4fdc849eddf33f6be746ffc8"}
Sep 17 14:22:51.402627 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:51.402560    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-9b8115/test-model-9b8115-kserve-7896bbcf7c-x294s" podStartSLOduration=1.444796991 podStartE2EDuration="2.402544258s" totalImagesPullingTime="957.747267ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:22:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:22:51.401721193 +0000 UTC m=+1072.403264678" watchObservedRunningTime="2026-09-17 14:22:51.402544258 +0000 UTC m=+1072.404087731"
Sep 17 14:22:54.789386 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:54.789302    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:54.860930 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:54.860895    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:55.023714 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:55.023675    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:55.326700 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:55.326663    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt"]
Sep 17 14:22:55.330298 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:55.330271    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt"
Sep 17 14:22:55.333460 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:55.333437    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-2137c903-openshift-default-dockercfg-rshtr\""
Sep 17 14:22:55.343790 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:55.343758    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt"]
Sep 17 14:22:55.370596 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:55.370545    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:55.420506 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:55.420462    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvgz8\" (UniqueName: \"kubernetes.io/projected/0f93aaa8-c6b7-404e-bb58-5b0786be570f-kube-api-access-dvgz8\") pod \"e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt\" (UID: \"0f93aaa8-c6b7-404e-bb58-5b0786be570f\") " pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt"
Sep 17 14:22:55.420739 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:55.420517    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f93aaa8-c6b7-404e-bb58-5b0786be570f-istio-token\") pod \"e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt\" (UID: \"0f93aaa8-c6b7-404e-bb58-5b0786be570f\") " pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt"
Sep 17 14:22:55.420739 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:55.420571    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f93aaa8-c6b7-404e-bb58-5b0786be570f-istio-envoy\") pod \"e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt\" (UID: \"0f93aaa8-c6b7-404e-bb58-5b0786be570f\") " pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt"
Sep 17 14:22:55.420739 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:55.420653    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f93aaa8-c6b7-404e-bb58-5b0786be570f-istiod-ca-cert\") pod \"e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt\" (UID: \"0f93aaa8-c6b7-404e-bb58-5b0786be570f\") " pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt"
Sep 17 14:22:55.420739 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:55.420690    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f93aaa8-c6b7-404e-bb58-5b0786be570f-credential-socket\") pod \"e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt\" (UID: \"0f93aaa8-c6b7-404e-bb58-5b0786be570f\") " pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt"
Sep 17 14:22:55.420739 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:55.420709    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f93aaa8-c6b7-404e-bb58-5b0786be570f-istio-podinfo\") pod \"e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt\" (UID: \"0f93aaa8-c6b7-404e-bb58-5b0786be570f\") " pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt"
Sep 17 14:22:55.420949 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:55.420756    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f93aaa8-c6b7-404e-bb58-5b0786be570f-workload-certs\") pod \"e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt\" (UID: \"0f93aaa8-c6b7-404e-bb58-5b0786be570f\") " pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt"
Sep 17 14:22:55.420949 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:55.420806    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f93aaa8-c6b7-404e-bb58-5b0786be570f-istio-data\") pod \"e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt\" (UID: \"0f93aaa8-c6b7-404e-bb58-5b0786be570f\") " pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt"
Sep 17 14:22:55.420949 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:55.420885    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f93aaa8-c6b7-404e-bb58-5b0786be570f-workload-socket\") pod \"e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt\" (UID: \"0f93aaa8-c6b7-404e-bb58-5b0786be570f\") " pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt"
Sep 17 14:22:55.470056 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:55.470020    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:22:55.521498 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:55.521461    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f93aaa8-c6b7-404e-bb58-5b0786be570f-workload-certs\") pod \"e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt\" (UID: \"0f93aaa8-c6b7-404e-bb58-5b0786be570f\") " pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt"
Sep 17 14:22:55.521720 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:55.521563    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f93aaa8-c6b7-404e-bb58-5b0786be570f-istio-data\") pod \"e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt\" (UID: \"0f93aaa8-c6b7-404e-bb58-5b0786be570f\") " pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt"
Sep 17 14:22:55.521720 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:55.521670    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f93aaa8-c6b7-404e-bb58-5b0786be570f-workload-socket\") pod \"e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt\" (UID: \"0f93aaa8-c6b7-404e-bb58-5b0786be570f\") " pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt"
Sep 17 14:22:55.521860 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:55.521725    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvgz8\" (UniqueName: \"kubernetes.io/projected/0f93aaa8-c6b7-404e-bb58-5b0786be570f-kube-api-access-dvgz8\") pod \"e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt\" (UID: \"0f93aaa8-c6b7-404e-bb58-5b0786be570f\") " pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt"
Sep 17 14:22:55.521860 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:55.521770    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f93aaa8-c6b7-404e-bb58-5b0786be570f-istio-token\") pod \"e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt\" (UID: \"0f93aaa8-c6b7-404e-bb58-5b0786be570f\") " pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt"
Sep 17 14:22:55.521860 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:55.521824    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f93aaa8-c6b7-404e-bb58-5b0786be570f-istio-envoy\") pod \"e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt\" (UID: \"0f93aaa8-c6b7-404e-bb58-5b0786be570f\") " pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt"
Sep 17 14:22:55.522022 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:55.521864    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f93aaa8-c6b7-404e-bb58-5b0786be570f-istiod-ca-cert\") pod \"e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt\" (UID: \"0f93aaa8-c6b7-404e-bb58-5b0786be570f\") " pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt"
Sep 17 14:22:55.522022 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:55.521896    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f93aaa8-c6b7-404e-bb58-5b0786be570f-credential-socket\") pod \"e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt\" (UID: \"0f93aaa8-c6b7-404e-bb58-5b0786be570f\") " pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt"
Sep 17 14:22:55.522022 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:55.521923    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f93aaa8-c6b7-404e-bb58-5b0786be570f-istio-podinfo\") pod \"e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt\" (UID: \"0f93aaa8-c6b7-404e-bb58-5b0786be570f\") " pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt"
Sep 17 14:22:55.522189 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:55.522017    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f93aaa8-c6b7-404e-bb58-5b0786be570f-istio-data\") pod \"e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt\" (UID: \"0f93aaa8-c6b7-404e-bb58-5b0786be570f\") " pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt"
Sep 17 14:22:55.522189 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:55.522048    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f93aaa8-c6b7-404e-bb58-5b0786be570f-workload-certs\") pod \"e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt\" (UID: \"0f93aaa8-c6b7-404e-bb58-5b0786be570f\") " pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt"
Sep 17 14:22:55.522189 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:55.522077    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f93aaa8-c6b7-404e-bb58-5b0786be570f-workload-socket\") pod \"e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt\" (UID: \"0f93aaa8-c6b7-404e-bb58-5b0786be570f\") " pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt"
Sep 17 14:22:55.522347 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:55.522306    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f93aaa8-c6b7-404e-bb58-5b0786be570f-credential-socket\") pod \"e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt\" (UID: \"0f93aaa8-c6b7-404e-bb58-5b0786be570f\") " pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt"
Sep 17 14:22:55.522683 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:55.522663    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f93aaa8-c6b7-404e-bb58-5b0786be570f-istiod-ca-cert\") pod \"e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt\" (UID: \"0f93aaa8-c6b7-404e-bb58-5b0786be570f\") " pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt"
Sep 17 14:22:55.524247 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:55.524223    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f93aaa8-c6b7-404e-bb58-5b0786be570f-istio-envoy\") pod \"e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt\" (UID: \"0f93aaa8-c6b7-404e-bb58-5b0786be570f\") " pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt"
Sep 17 14:22:55.524330 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:55.524290    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f93aaa8-c6b7-404e-bb58-5b0786be570f-istio-podinfo\") pod \"e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt\" (UID: \"0f93aaa8-c6b7-404e-bb58-5b0786be570f\") " pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt"
Sep 17 14:22:55.532139 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:55.532088    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvgz8\" (UniqueName: \"kubernetes.io/projected/0f93aaa8-c6b7-404e-bb58-5b0786be570f-kube-api-access-dvgz8\") pod \"e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt\" (UID: \"0f93aaa8-c6b7-404e-bb58-5b0786be570f\") " pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt"
Sep 17 14:22:55.532424 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:55.532404    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f93aaa8-c6b7-404e-bb58-5b0786be570f-istio-token\") pod \"e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt\" (UID: \"0f93aaa8-c6b7-404e-bb58-5b0786be570f\") " pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt"
Sep 17 14:22:55.642495 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:55.642411    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt"
Sep 17 14:22:55.921933 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:55.921901    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt"]
Sep 17 14:22:55.923569 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:22:55.923529    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f93aaa8_c6b7_404e_bb58_5b0786be570f.slice/crio-35b9c77c8ece504657a9659f33b6f25b73971e10280b244c28c38f7a574d8764 WatchSource:0}: Error finding container 35b9c77c8ece504657a9659f33b6f25b73971e10280b244c28c38f7a574d8764: Status 404 returned error can't find the container with id 35b9c77c8ece504657a9659f33b6f25b73971e10280b244c28c38f7a574d8764
Sep 17 14:22:55.926017 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:55.925983    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:22:55.926091 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:55.926060    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:22:55.926126 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:55.926097    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:22:56.416341 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:56.416298    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt" event={"ID":"0f93aaa8-c6b7-404e-bb58-5b0786be570f","Type":"ContainerStarted","Data":"1ccafe42e2c0b85fe3591433652d8ab28c89962bac7ea1439186cd22f2dd398f"}
Sep 17 14:22:56.416497 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:56.416390    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt" event={"ID":"0f93aaa8-c6b7-404e-bb58-5b0786be570f","Type":"ContainerStarted","Data":"35b9c77c8ece504657a9659f33b6f25b73971e10280b244c28c38f7a574d8764"}
Sep 17 14:22:56.442342 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:56.442282    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt" podStartSLOduration=1.4422656520000001 podStartE2EDuration="1.442265652s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:22:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:22:56.44071605 +0000 UTC m=+1077.442259526" watchObservedRunningTime="2026-09-17 14:22:56.442265652 +0000 UTC m=+1077.443809130"
Sep 17 14:22:56.643299 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:56.643260    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt"
Sep 17 14:22:56.649817 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:56.649785    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt"
Sep 17 14:22:57.426138 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:57.426105    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt"
Sep 17 14:22:57.427435 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:57.427393    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt"
Sep 17 14:22:59.706512 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:59.706470    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-9b8115/test-model-9b8115-kserve-7896bbcf7c-x294s"
Sep 17 14:22:59.706929 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:59.706706    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-9b8115/test-model-9b8115-kserve-7896bbcf7c-x294s"
Sep 17 14:22:59.719581 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:22:59.719556    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-9b8115/test-model-9b8115-kserve-7896bbcf7c-x294s"
Sep 17 14:23:00.465661 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:00.465630    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-9b8115/test-model-9b8115-kserve-7896bbcf7c-x294s"
Sep 17 14:23:00.967382 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:00.967345    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:01.550222 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:01.550186    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:01.653311 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:01.653277    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:01.754553 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:01.754516    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:02.448309 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:02.448275    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:03.243334 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:03.243295    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:04.125969 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:04.125930    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:04.898447 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:04.898414    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:05.666012 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:05.665979    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:05.738177 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:05.738136    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:06.223855 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:06.223800    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:06.296749 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:06.296719    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:08.422167 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:08.422122    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-64585d4699-8csw4" podUID="b34d9da8-57dc-43b0-9d2a-3894a00a6df9" containerName="console" containerID="cri-o://b4cb96a7ef8d4772b92383548e0e9ab49896f936b955969df03caba5592eff9e" gracePeriod=15
Sep 17 14:23:08.545166 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:08.545134    2582 generic.go:356] "Generic (PLEG): container finished" podID="b34d9da8-57dc-43b0-9d2a-3894a00a6df9" containerID="b4cb96a7ef8d4772b92383548e0e9ab49896f936b955969df03caba5592eff9e" exitCode=0
Sep 17 14:23:08.545332 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:08.545216    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64585d4699-8csw4" event={"ID":"b34d9da8-57dc-43b0-9d2a-3894a00a6df9","Type":"ContainerDied","Data":"b4cb96a7ef8d4772b92383548e0e9ab49896f936b955969df03caba5592eff9e"}
Sep 17 14:23:08.692648 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:08.692617    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64585d4699-8csw4"
Sep 17 14:23:08.758463 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:08.758431    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b34d9da8-57dc-43b0-9d2a-3894a00a6df9-console-config\") pod \"b34d9da8-57dc-43b0-9d2a-3894a00a6df9\" (UID: \"b34d9da8-57dc-43b0-9d2a-3894a00a6df9\") "
Sep 17 14:23:08.758688 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:08.758484    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b34d9da8-57dc-43b0-9d2a-3894a00a6df9-trusted-ca-bundle\") pod \"b34d9da8-57dc-43b0-9d2a-3894a00a6df9\" (UID: \"b34d9da8-57dc-43b0-9d2a-3894a00a6df9\") "
Sep 17 14:23:08.758688 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:08.758504    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b34d9da8-57dc-43b0-9d2a-3894a00a6df9-service-ca\") pod \"b34d9da8-57dc-43b0-9d2a-3894a00a6df9\" (UID: \"b34d9da8-57dc-43b0-9d2a-3894a00a6df9\") "
Sep 17 14:23:08.758688 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:08.758536    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-smzj2\" (UniqueName: \"kubernetes.io/projected/b34d9da8-57dc-43b0-9d2a-3894a00a6df9-kube-api-access-smzj2\") pod \"b34d9da8-57dc-43b0-9d2a-3894a00a6df9\" (UID: \"b34d9da8-57dc-43b0-9d2a-3894a00a6df9\") "
Sep 17 14:23:08.758688 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:08.758575    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b34d9da8-57dc-43b0-9d2a-3894a00a6df9-console-serving-cert\") pod \"b34d9da8-57dc-43b0-9d2a-3894a00a6df9\" (UID: \"b34d9da8-57dc-43b0-9d2a-3894a00a6df9\") "
Sep 17 14:23:08.758688 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:08.758601    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b34d9da8-57dc-43b0-9d2a-3894a00a6df9-oauth-serving-cert\") pod \"b34d9da8-57dc-43b0-9d2a-3894a00a6df9\" (UID: \"b34d9da8-57dc-43b0-9d2a-3894a00a6df9\") "
Sep 17 14:23:08.759050 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:08.758733    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b34d9da8-57dc-43b0-9d2a-3894a00a6df9-console-oauth-config\") pod \"b34d9da8-57dc-43b0-9d2a-3894a00a6df9\" (UID: \"b34d9da8-57dc-43b0-9d2a-3894a00a6df9\") "
Sep 17 14:23:08.759050 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:08.758915    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b34d9da8-57dc-43b0-9d2a-3894a00a6df9-service-ca" (OuterVolumeSpecName: "service-ca") pod "b34d9da8-57dc-43b0-9d2a-3894a00a6df9" (UID: "b34d9da8-57dc-43b0-9d2a-3894a00a6df9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:23:08.759050 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:08.758924    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b34d9da8-57dc-43b0-9d2a-3894a00a6df9-console-config" (OuterVolumeSpecName: "console-config") pod "b34d9da8-57dc-43b0-9d2a-3894a00a6df9" (UID: "b34d9da8-57dc-43b0-9d2a-3894a00a6df9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:23:08.759050 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:08.758942    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b34d9da8-57dc-43b0-9d2a-3894a00a6df9-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b34d9da8-57dc-43b0-9d2a-3894a00a6df9" (UID: "b34d9da8-57dc-43b0-9d2a-3894a00a6df9"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:23:08.759255 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:08.759134    2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b34d9da8-57dc-43b0-9d2a-3894a00a6df9-console-config\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:23:08.759255 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:08.759150    2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b34d9da8-57dc-43b0-9d2a-3894a00a6df9-trusted-ca-bundle\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:23:08.759255 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:08.759159    2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b34d9da8-57dc-43b0-9d2a-3894a00a6df9-service-ca\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:23:08.759255 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:08.759230    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b34d9da8-57dc-43b0-9d2a-3894a00a6df9-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b34d9da8-57dc-43b0-9d2a-3894a00a6df9" (UID: "b34d9da8-57dc-43b0-9d2a-3894a00a6df9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:23:08.760984 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:08.760959    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b34d9da8-57dc-43b0-9d2a-3894a00a6df9-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b34d9da8-57dc-43b0-9d2a-3894a00a6df9" (UID: "b34d9da8-57dc-43b0-9d2a-3894a00a6df9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:23:08.761451 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:08.761430    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b34d9da8-57dc-43b0-9d2a-3894a00a6df9-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b34d9da8-57dc-43b0-9d2a-3894a00a6df9" (UID: "b34d9da8-57dc-43b0-9d2a-3894a00a6df9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:23:08.761522 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:08.761458    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b34d9da8-57dc-43b0-9d2a-3894a00a6df9-kube-api-access-smzj2" (OuterVolumeSpecName: "kube-api-access-smzj2") pod "b34d9da8-57dc-43b0-9d2a-3894a00a6df9" (UID: "b34d9da8-57dc-43b0-9d2a-3894a00a6df9"). InnerVolumeSpecName "kube-api-access-smzj2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:23:08.860276 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:08.860236    2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b34d9da8-57dc-43b0-9d2a-3894a00a6df9-console-oauth-config\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:23:08.860276 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:08.860267    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-smzj2\" (UniqueName: \"kubernetes.io/projected/b34d9da8-57dc-43b0-9d2a-3894a00a6df9-kube-api-access-smzj2\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:23:08.860276 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:08.860277    2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b34d9da8-57dc-43b0-9d2a-3894a00a6df9-console-serving-cert\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:23:08.860276 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:08.860285    2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b34d9da8-57dc-43b0-9d2a-3894a00a6df9-oauth-serving-cert\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:23:09.217907 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:09.217872    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:09.303424 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:09.303379    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:09.552415 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:09.552382    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64585d4699-8csw4"
Sep 17 14:23:09.552849 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:09.552408    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64585d4699-8csw4" event={"ID":"b34d9da8-57dc-43b0-9d2a-3894a00a6df9","Type":"ContainerDied","Data":"43ff7651e67861a03b52f7fbe8880edd515cd310b0c4a2f0b1edb08536256460"}
Sep 17 14:23:09.552849 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:09.552473    2582 scope.go:117] "RemoveContainer" containerID="b4cb96a7ef8d4772b92383548e0e9ab49896f936b955969df03caba5592eff9e"
Sep 17 14:23:09.577466 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:09.577435    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64585d4699-8csw4"]
Sep 17 14:23:09.580883 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:09.580859    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-64585d4699-8csw4"]
Sep 17 14:23:11.496086 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:11.496049    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b34d9da8-57dc-43b0-9d2a-3894a00a6df9" path="/var/lib/kubelet/pods/b34d9da8-57dc-43b0-9d2a-3894a00a6df9/volumes"
Sep 17 14:23:19.226352 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:19.226320    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:27.886173 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:27.886124    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:28.671438 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:28.671399    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:29.388225 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:29.388192    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:30.242368 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:30.242331    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:31.151465 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:31.151428    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:31.690568 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:31.690535    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:31.959411 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:31.959374    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:32.437748 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:32.437640    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:32.615329 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:32.615296    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:32.784896 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:32.784857    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:32.883760 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:32.883724    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:37.724303 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:37.724267    2582 generic.go:356] "Generic (PLEG): container finished" podID="0f93aaa8-c6b7-404e-bb58-5b0786be570f" containerID="1ccafe42e2c0b85fe3591433652d8ab28c89962bac7ea1439186cd22f2dd398f" exitCode=0
Sep 17 14:23:37.724757 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:37.724311    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt" event={"ID":"0f93aaa8-c6b7-404e-bb58-5b0786be570f","Type":"ContainerDied","Data":"1ccafe42e2c0b85fe3591433652d8ab28c89962bac7ea1439186cd22f2dd398f"}
Sep 17 14:23:37.724757 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:37.724748    2582 scope.go:117] "RemoveContainer" containerID="1ccafe42e2c0b85fe3591433652d8ab28c89962bac7ea1439186cd22f2dd398f"
Sep 17 14:23:37.725970 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:37.725941    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:23:37.726039 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:37.726010    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:23:37.726076 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:37.726039    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:23:38.735539 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:38.735503    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt" event={"ID":"0f93aaa8-c6b7-404e-bb58-5b0786be570f","Type":"ContainerStarted","Data":"a9a107d5139c697ae6bf30bf25d477ee48284f0f214b2d5c8def1759ed009aaf"}
Sep 17 14:23:39.223248 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:39.223213    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:39.342876 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:39.342836    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:39.517038 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:39.516958    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2137c903-xlbpd"]
Sep 17 14:23:39.517433 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:39.517421    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b34d9da8-57dc-43b0-9d2a-3894a00a6df9" containerName="console"
Sep 17 14:23:39.517482 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:39.517435    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b34d9da8-57dc-43b0-9d2a-3894a00a6df9" containerName="console"
Sep 17 14:23:39.517521 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:39.517503    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b34d9da8-57dc-43b0-9d2a-3894a00a6df9" containerName="console"
Sep 17 14:23:39.520643 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:39.520627    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2137c903-xlbpd"
Sep 17 14:23:39.523312 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:39.523289    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ddwrn\""
Sep 17 14:23:39.535301 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:39.535274    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2137c903-xlbpd"]
Sep 17 14:23:39.591065 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:39.591029    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52cd7e41-4c95-4555-8b53-48cf0b51ac62-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2137c903-xlbpd\" (UID: \"52cd7e41-4c95-4555-8b53-48cf0b51ac62\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2137c903-xlbpd"
Sep 17 14:23:39.646677 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:39.645785    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt"
Sep 17 14:23:39.653634 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:39.652771    2582 patch_prober.go:28] interesting pod/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Sep 17 14:23:39.653634 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:39.652851    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt" podUID="0f93aaa8-c6b7-404e-bb58-5b0786be570f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Sep 17 14:23:39.692219 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:39.692184    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52cd7e41-4c95-4555-8b53-48cf0b51ac62-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2137c903-xlbpd\" (UID: \"52cd7e41-4c95-4555-8b53-48cf0b51ac62\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2137c903-xlbpd"
Sep 17 14:23:39.692885 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:39.692863    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52cd7e41-4c95-4555-8b53-48cf0b51ac62-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2137c903-xlbpd\" (UID: \"52cd7e41-4c95-4555-8b53-48cf0b51ac62\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2137c903-xlbpd"
Sep 17 14:23:39.834352 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:39.834239    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2137c903-xlbpd"
Sep 17 14:23:40.126924 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:40.126839    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2137c903-xlbpd"]
Sep 17 14:23:40.331941 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:40.331901    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:40.643156 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:40.643106    2582 patch_prober.go:28] interesting pod/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Sep 17 14:23:40.643367 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:40.643185    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt" podUID="0f93aaa8-c6b7-404e-bb58-5b0786be570f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Sep 17 14:23:40.756313 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:40.756203    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2137c903-xlbpd" event={"ID":"52cd7e41-4c95-4555-8b53-48cf0b51ac62","Type":"ContainerStarted","Data":"391e397e164a0fd2fb4805a77466f49c3ea4ab9eb459bd6ff53ee1589049ee50"}
Sep 17 14:23:40.756313 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:40.756277    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2137c903-xlbpd" event={"ID":"52cd7e41-4c95-4555-8b53-48cf0b51ac62","Type":"ContainerStarted","Data":"97013e7b2acb57af68febbd88f1a4f93e1bac81d35c289cfc63ee438b4638eb9"}
Sep 17 14:23:40.777411 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:40.777352    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2137c903-xlbpd" podStartSLOduration=1.777332466 podStartE2EDuration="1.777332466s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:23:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:23:40.77696899 +0000 UTC m=+1121.778512464" watchObservedRunningTime="2026-09-17 14:23:40.777332466 +0000 UTC m=+1121.778875942"
Sep 17 14:23:40.991688 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:40.991653    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:41.644044 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:41.643590    2582 patch_prober.go:28] interesting pod/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Sep 17 14:23:41.644044 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:41.643673    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt" podUID="0f93aaa8-c6b7-404e-bb58-5b0786be570f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Sep 17 14:23:41.689896 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:41.689854    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:41.763967 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:41.763933    2582 generic.go:356] "Generic (PLEG): container finished" podID="52cd7e41-4c95-4555-8b53-48cf0b51ac62" containerID="391e397e164a0fd2fb4805a77466f49c3ea4ab9eb459bd6ff53ee1589049ee50" exitCode=0
Sep 17 14:23:41.764177 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:41.763997    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2137c903-xlbpd" event={"ID":"52cd7e41-4c95-4555-8b53-48cf0b51ac62","Type":"ContainerDied","Data":"391e397e164a0fd2fb4805a77466f49c3ea4ab9eb459bd6ff53ee1589049ee50"}
Sep 17 14:23:42.379597 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:42.379559    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:42.425791 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:42.425756    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt"
Sep 17 14:23:42.643768 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:42.643671    2582 patch_prober.go:28] interesting pod/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Sep 17 14:23:42.644050 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:42.644015    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt" podUID="0f93aaa8-c6b7-404e-bb58-5b0786be570f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Sep 17 14:23:42.952139 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:42.951781    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2137c903-xlbpd"
Sep 17 14:23:43.056630 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:43.055951    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52cd7e41-4c95-4555-8b53-48cf0b51ac62-maas-api-service-ca\") pod \"52cd7e41-4c95-4555-8b53-48cf0b51ac62\" (UID: \"52cd7e41-4c95-4555-8b53-48cf0b51ac62\") "
Sep 17 14:23:43.056831 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:43.056779    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52cd7e41-4c95-4555-8b53-48cf0b51ac62-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "52cd7e41-4c95-4555-8b53-48cf0b51ac62" (UID: "52cd7e41-4c95-4555-8b53-48cf0b51ac62"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:23:43.099510 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:43.099468    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:43.157366 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:43.157239    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52cd7e41-4c95-4555-8b53-48cf0b51ac62-maas-api-service-ca\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:23:43.643036 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:43.642992    2582 patch_prober.go:28] interesting pod/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Sep 17 14:23:43.643444 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:43.643053    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt" podUID="0f93aaa8-c6b7-404e-bb58-5b0786be570f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Sep 17 14:23:43.750243 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:43.750203    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:43.778832 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:43.778735    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2137c903-xlbpd"
Sep 17 14:23:43.778832 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:43.778751    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2137c903-xlbpd" event={"ID":"52cd7e41-4c95-4555-8b53-48cf0b51ac62","Type":"ContainerDied","Data":"97013e7b2acb57af68febbd88f1a4f93e1bac81d35c289cfc63ee438b4638eb9"}
Sep 17 14:23:43.778832 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:43.778797    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="97013e7b2acb57af68febbd88f1a4f93e1bac81d35c289cfc63ee438b4638eb9"
Sep 17 14:23:44.545265 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:44.545230    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:44.647302 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:44.647272    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt"
Sep 17 14:23:44.648438 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:44.648416    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt"
Sep 17 14:23:50.293495 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:50.293457    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:50.398267 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:50.398233    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2137c903-xlbpd"]
Sep 17 14:23:50.401448 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:50.401421    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2137c903-xlbpd"]
Sep 17 14:23:50.430483 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:50.430441    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:50.602166 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:50.602079    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:50.860986 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:50.860912    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:50.970223 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:50.970188    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:51.494931 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:51.494900    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52cd7e41-4c95-4555-8b53-48cf0b51ac62" path="/var/lib/kubelet/pods/52cd7e41-4c95-4555-8b53-48cf0b51ac62/volumes"
Sep 17 14:23:51.748733 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:51.748655    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:52.472895 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:52.472859    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:53.137965 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:53.137926    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:53.546652 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:53.546601    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-68183c-99r4r"]
Sep 17 14:23:53.547107 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:53.547094    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52cd7e41-4c95-4555-8b53-48cf0b51ac62" containerName="revoke-keys"
Sep 17 14:23:53.547146 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:53.547108    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="52cd7e41-4c95-4555-8b53-48cf0b51ac62" containerName="revoke-keys"
Sep 17 14:23:53.547201 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:53.547192    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="52cd7e41-4c95-4555-8b53-48cf0b51ac62" containerName="revoke-keys"
Sep 17 14:23:53.550693 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:53.550677    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-68183c-99r4r"
Sep 17 14:23:53.553049 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:53.553026    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ddwrn\""
Sep 17 14:23:53.560270 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:53.560248    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-68183c-99r4r"]
Sep 17 14:23:53.669663 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:53.669630    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/933f531c-79ea-4701-b112-e7f801d2ed40-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-68183c-99r4r\" (UID: \"933f531c-79ea-4701-b112-e7f801d2ed40\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-68183c-99r4r"
Sep 17 14:23:53.771110 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:53.771071    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/933f531c-79ea-4701-b112-e7f801d2ed40-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-68183c-99r4r\" (UID: \"933f531c-79ea-4701-b112-e7f801d2ed40\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-68183c-99r4r"
Sep 17 14:23:53.771711 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:53.771691    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/933f531c-79ea-4701-b112-e7f801d2ed40-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-68183c-99r4r\" (UID: \"933f531c-79ea-4701-b112-e7f801d2ed40\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-68183c-99r4r"
Sep 17 14:23:53.863078 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:53.862994    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-68183c-99r4r"
Sep 17 14:23:53.983359 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:53.983325    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:54.139865 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:54.139834    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-68183c-99r4r"]
Sep 17 14:23:54.141184 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:23:54.141139    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod933f531c_79ea_4701_b112_e7f801d2ed40.slice/crio-b2be4afc351c3b69828d4f6cd72605e8b03d1b8993b96c3bd023f7d873edcdba WatchSource:0}: Error finding container b2be4afc351c3b69828d4f6cd72605e8b03d1b8993b96c3bd023f7d873edcdba: Status 404 returned error can't find the container with id b2be4afc351c3b69828d4f6cd72605e8b03d1b8993b96c3bd023f7d873edcdba
Sep 17 14:23:54.760472 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:54.760441    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:54.852579 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:54.852481    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-68183c-99r4r" event={"ID":"933f531c-79ea-4701-b112-e7f801d2ed40","Type":"ContainerStarted","Data":"e5423e518fd285e9195d0b6f52909a9289a88c94f8cb0047542316cd72631f64"}
Sep 17 14:23:54.852579 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:54.852542    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-68183c-99r4r" event={"ID":"933f531c-79ea-4701-b112-e7f801d2ed40","Type":"ContainerStarted","Data":"b2be4afc351c3b69828d4f6cd72605e8b03d1b8993b96c3bd023f7d873edcdba"}
Sep 17 14:23:54.874836 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:54.874785    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-68183c-99r4r" podStartSLOduration=1.874768016 podStartE2EDuration="1.874768016s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:23:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:23:54.873196224 +0000 UTC m=+1135.874739699" watchObservedRunningTime="2026-09-17 14:23:54.874768016 +0000 UTC m=+1135.876311489"
Sep 17 14:23:55.441686 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:55.441646    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:55.860507 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:55.860468    2582 generic.go:356] "Generic (PLEG): container finished" podID="933f531c-79ea-4701-b112-e7f801d2ed40" containerID="e5423e518fd285e9195d0b6f52909a9289a88c94f8cb0047542316cd72631f64" exitCode=0
Sep 17 14:23:55.860735 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:55.860557    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-68183c-99r4r" event={"ID":"933f531c-79ea-4701-b112-e7f801d2ed40","Type":"ContainerDied","Data":"e5423e518fd285e9195d0b6f52909a9289a88c94f8cb0047542316cd72631f64"}
Sep 17 14:23:56.165440 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:56.165326    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:56.840601 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:56.840554    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:56.964546 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:56.964508    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:57.015171 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:57.015147    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-68183c-99r4r"
Sep 17 14:23:57.038127 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:57.038080    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:57.081370 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:57.081335    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt"]
Sep 17 14:23:57.081637 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:57.081574    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt" podUID="0f93aaa8-c6b7-404e-bb58-5b0786be570f" containerName="istio-proxy" containerID="cri-o://a9a107d5139c697ae6bf30bf25d477ee48284f0f214b2d5c8def1759ed009aaf" gracePeriod=30
Sep 17 14:23:57.146095 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:57.146004    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:57.207152 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:57.207100    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/933f531c-79ea-4701-b112-e7f801d2ed40-maas-api-service-ca\") pod \"933f531c-79ea-4701-b112-e7f801d2ed40\" (UID: \"933f531c-79ea-4701-b112-e7f801d2ed40\") "
Sep 17 14:23:57.207715 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:57.207684    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/933f531c-79ea-4701-b112-e7f801d2ed40-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "933f531c-79ea-4701-b112-e7f801d2ed40" (UID: "933f531c-79ea-4701-b112-e7f801d2ed40"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:23:57.308536 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:57.308505    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/933f531c-79ea-4701-b112-e7f801d2ed40-maas-api-service-ca\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:23:57.474008 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:57.473968    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:57.568354 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:57.568321    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:57.691570 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:57.691536    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:57.873889 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:57.873815    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-68183c-99r4r"
Sep 17 14:23:57.874235 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:57.873818    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-68183c-99r4r" event={"ID":"933f531c-79ea-4701-b112-e7f801d2ed40","Type":"ContainerDied","Data":"b2be4afc351c3b69828d4f6cd72605e8b03d1b8993b96c3bd023f7d873edcdba"}
Sep 17 14:23:57.874235 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:57.873936    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b2be4afc351c3b69828d4f6cd72605e8b03d1b8993b96c3bd023f7d873edcdba"
Sep 17 14:23:57.915719 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:57.915572    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:59.110029 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:59.109992    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:23:59.650075 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:59.650031    2582 patch_prober.go:28] interesting pod/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 17 14:23:59.650251 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:23:59.650086    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt" podUID="0f93aaa8-c6b7-404e-bb58-5b0786be570f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 14:24:02.347216 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:02.347151    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt"
Sep 17 14:24:02.355746 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:02.355721    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dvgz8\" (UniqueName: \"kubernetes.io/projected/0f93aaa8-c6b7-404e-bb58-5b0786be570f-kube-api-access-dvgz8\") pod \"0f93aaa8-c6b7-404e-bb58-5b0786be570f\" (UID: \"0f93aaa8-c6b7-404e-bb58-5b0786be570f\") "
Sep 17 14:24:02.355835 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:02.355756    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f93aaa8-c6b7-404e-bb58-5b0786be570f-istio-token\") pod \"0f93aaa8-c6b7-404e-bb58-5b0786be570f\" (UID: \"0f93aaa8-c6b7-404e-bb58-5b0786be570f\") "
Sep 17 14:24:02.355835 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:02.355795    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f93aaa8-c6b7-404e-bb58-5b0786be570f-istio-data\") pod \"0f93aaa8-c6b7-404e-bb58-5b0786be570f\" (UID: \"0f93aaa8-c6b7-404e-bb58-5b0786be570f\") "
Sep 17 14:24:02.355835 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:02.355819    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f93aaa8-c6b7-404e-bb58-5b0786be570f-workload-socket\") pod \"0f93aaa8-c6b7-404e-bb58-5b0786be570f\" (UID: \"0f93aaa8-c6b7-404e-bb58-5b0786be570f\") "
Sep 17 14:24:02.355990 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:02.355847    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f93aaa8-c6b7-404e-bb58-5b0786be570f-istio-envoy\") pod \"0f93aaa8-c6b7-404e-bb58-5b0786be570f\" (UID: \"0f93aaa8-c6b7-404e-bb58-5b0786be570f\") "
Sep 17 14:24:02.355990 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:02.355884    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f93aaa8-c6b7-404e-bb58-5b0786be570f-istiod-ca-cert\") pod \"0f93aaa8-c6b7-404e-bb58-5b0786be570f\" (UID: \"0f93aaa8-c6b7-404e-bb58-5b0786be570f\") "
Sep 17 14:24:02.356095 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:02.355990    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f93aaa8-c6b7-404e-bb58-5b0786be570f-credential-socket\") pod \"0f93aaa8-c6b7-404e-bb58-5b0786be570f\" (UID: \"0f93aaa8-c6b7-404e-bb58-5b0786be570f\") "
Sep 17 14:24:02.356095 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:02.356028    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f93aaa8-c6b7-404e-bb58-5b0786be570f-istio-podinfo\") pod \"0f93aaa8-c6b7-404e-bb58-5b0786be570f\" (UID: \"0f93aaa8-c6b7-404e-bb58-5b0786be570f\") "
Sep 17 14:24:02.356095 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:02.356078    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f93aaa8-c6b7-404e-bb58-5b0786be570f-workload-certs\") pod \"0f93aaa8-c6b7-404e-bb58-5b0786be570f\" (UID: \"0f93aaa8-c6b7-404e-bb58-5b0786be570f\") "
Sep 17 14:24:02.356243 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:02.356165    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f93aaa8-c6b7-404e-bb58-5b0786be570f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0f93aaa8-c6b7-404e-bb58-5b0786be570f" (UID: "0f93aaa8-c6b7-404e-bb58-5b0786be570f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:24:02.356371 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:02.356321    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f93aaa8-c6b7-404e-bb58-5b0786be570f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0f93aaa8-c6b7-404e-bb58-5b0786be570f" (UID: "0f93aaa8-c6b7-404e-bb58-5b0786be570f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:24:02.356457 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:02.356392    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f93aaa8-c6b7-404e-bb58-5b0786be570f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0f93aaa8-c6b7-404e-bb58-5b0786be570f" (UID: "0f93aaa8-c6b7-404e-bb58-5b0786be570f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:24:02.356457 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:02.356420    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f93aaa8-c6b7-404e-bb58-5b0786be570f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0f93aaa8-c6b7-404e-bb58-5b0786be570f" (UID: "0f93aaa8-c6b7-404e-bb58-5b0786be570f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:24:02.356574 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:02.356457    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f93aaa8-c6b7-404e-bb58-5b0786be570f-istio-data" (OuterVolumeSpecName: "istio-data") pod "0f93aaa8-c6b7-404e-bb58-5b0786be570f" (UID: "0f93aaa8-c6b7-404e-bb58-5b0786be570f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:24:02.356574 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:02.356538    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f93aaa8-c6b7-404e-bb58-5b0786be570f-workload-socket\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:24:02.356574 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:02.356557    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f93aaa8-c6b7-404e-bb58-5b0786be570f-istiod-ca-cert\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:24:02.356574 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:02.356569    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f93aaa8-c6b7-404e-bb58-5b0786be570f-credential-socket\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:24:02.356772 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:02.356581    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f93aaa8-c6b7-404e-bb58-5b0786be570f-workload-certs\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:24:02.358242 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:02.358217    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f93aaa8-c6b7-404e-bb58-5b0786be570f-istio-token" (OuterVolumeSpecName: "istio-token") pod "0f93aaa8-c6b7-404e-bb58-5b0786be570f" (UID: "0f93aaa8-c6b7-404e-bb58-5b0786be570f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:24:02.358433 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:02.358412    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f93aaa8-c6b7-404e-bb58-5b0786be570f-kube-api-access-dvgz8" (OuterVolumeSpecName: "kube-api-access-dvgz8") pod "0f93aaa8-c6b7-404e-bb58-5b0786be570f" (UID: "0f93aaa8-c6b7-404e-bb58-5b0786be570f"). InnerVolumeSpecName "kube-api-access-dvgz8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:24:02.358549 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:02.358532    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f93aaa8-c6b7-404e-bb58-5b0786be570f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0f93aaa8-c6b7-404e-bb58-5b0786be570f" (UID: "0f93aaa8-c6b7-404e-bb58-5b0786be570f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:24:02.358841 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:02.358821    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0f93aaa8-c6b7-404e-bb58-5b0786be570f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0f93aaa8-c6b7-404e-bb58-5b0786be570f" (UID: "0f93aaa8-c6b7-404e-bb58-5b0786be570f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 14:24:02.457380 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:02.457344    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dvgz8\" (UniqueName: \"kubernetes.io/projected/0f93aaa8-c6b7-404e-bb58-5b0786be570f-kube-api-access-dvgz8\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:24:02.457380 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:02.457378    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f93aaa8-c6b7-404e-bb58-5b0786be570f-istio-token\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:24:02.457380 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:02.457388    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f93aaa8-c6b7-404e-bb58-5b0786be570f-istio-data\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:24:02.457597 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:02.457396    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f93aaa8-c6b7-404e-bb58-5b0786be570f-istio-envoy\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:24:02.457597 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:02.457404    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f93aaa8-c6b7-404e-bb58-5b0786be570f-istio-podinfo\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:24:02.908319 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:02.908276    2582 generic.go:356] "Generic (PLEG): container finished" podID="0f93aaa8-c6b7-404e-bb58-5b0786be570f" containerID="a9a107d5139c697ae6bf30bf25d477ee48284f0f214b2d5c8def1759ed009aaf" exitCode=0
Sep 17 14:24:02.908506 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:02.908331    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt"
Sep 17 14:24:02.908506 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:02.908332    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt" event={"ID":"0f93aaa8-c6b7-404e-bb58-5b0786be570f","Type":"ContainerDied","Data":"a9a107d5139c697ae6bf30bf25d477ee48284f0f214b2d5c8def1759ed009aaf"}
Sep 17 14:24:02.908506 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:02.908405    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt" event={"ID":"0f93aaa8-c6b7-404e-bb58-5b0786be570f","Type":"ContainerDied","Data":"35b9c77c8ece504657a9659f33b6f25b73971e10280b244c28c38f7a574d8764"}
Sep 17 14:24:02.908506 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:02.908437    2582 scope.go:117] "RemoveContainer" containerID="a9a107d5139c697ae6bf30bf25d477ee48284f0f214b2d5c8def1759ed009aaf"
Sep 17 14:24:02.919164 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:02.919143    2582 scope.go:117] "RemoveContainer" containerID="1ccafe42e2c0b85fe3591433652d8ab28c89962bac7ea1439186cd22f2dd398f"
Sep 17 14:24:02.928645 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:02.928590    2582 scope.go:117] "RemoveContainer" containerID="a9a107d5139c697ae6bf30bf25d477ee48284f0f214b2d5c8def1759ed009aaf"
Sep 17 14:24:02.929578 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:24:02.929340    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9a107d5139c697ae6bf30bf25d477ee48284f0f214b2d5c8def1759ed009aaf\": container with ID starting with a9a107d5139c697ae6bf30bf25d477ee48284f0f214b2d5c8def1759ed009aaf not found: ID does not exist" containerID="a9a107d5139c697ae6bf30bf25d477ee48284f0f214b2d5c8def1759ed009aaf"
Sep 17 14:24:02.929578 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:02.929403    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9a107d5139c697ae6bf30bf25d477ee48284f0f214b2d5c8def1759ed009aaf"} err="failed to get container status \"a9a107d5139c697ae6bf30bf25d477ee48284f0f214b2d5c8def1759ed009aaf\": rpc error: code = NotFound desc = could not find container \"a9a107d5139c697ae6bf30bf25d477ee48284f0f214b2d5c8def1759ed009aaf\": container with ID starting with a9a107d5139c697ae6bf30bf25d477ee48284f0f214b2d5c8def1759ed009aaf not found: ID does not exist"
Sep 17 14:24:02.929578 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:02.929432    2582 scope.go:117] "RemoveContainer" containerID="1ccafe42e2c0b85fe3591433652d8ab28c89962bac7ea1439186cd22f2dd398f"
Sep 17 14:24:02.929969 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:24:02.929888    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ccafe42e2c0b85fe3591433652d8ab28c89962bac7ea1439186cd22f2dd398f\": container with ID starting with 1ccafe42e2c0b85fe3591433652d8ab28c89962bac7ea1439186cd22f2dd398f not found: ID does not exist" containerID="1ccafe42e2c0b85fe3591433652d8ab28c89962bac7ea1439186cd22f2dd398f"
Sep 17 14:24:02.929969 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:02.929926    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ccafe42e2c0b85fe3591433652d8ab28c89962bac7ea1439186cd22f2dd398f"} err="failed to get container status \"1ccafe42e2c0b85fe3591433652d8ab28c89962bac7ea1439186cd22f2dd398f\": rpc error: code = NotFound desc = could not find container \"1ccafe42e2c0b85fe3591433652d8ab28c89962bac7ea1439186cd22f2dd398f\": container with ID starting with 1ccafe42e2c0b85fe3591433652d8ab28c89962bac7ea1439186cd22f2dd398f not found: ID does not exist"
Sep 17 14:24:02.932431 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:02.932408    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt"]
Sep 17 14:24:02.936747 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:02.936719    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-2137c903-openshift-default-c495b45cb-8fgzt"]
Sep 17 14:24:03.493971 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:03.493936    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f93aaa8-c6b7-404e-bb58-5b0786be570f" path="/var/lib/kubelet/pods/0f93aaa8-c6b7-404e-bb58-5b0786be570f/volumes"
Sep 17 14:24:03.798945 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:03.798859    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:03.909305 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:03.909261    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:04.015185 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:04.015149    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:04.147865 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:04.147783    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:04.613550 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:04.613519    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:04.741403 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:04.741372    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:04.770953 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:04.770914    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-68183c-99r4r"]
Sep 17 14:24:04.773900 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:04.773872    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-68183c-99r4r"]
Sep 17 14:24:04.855796 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:04.855747    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:05.191478 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:05.191443    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:05.335867 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:05.335831    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:05.495023 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:05.494992    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="933f531c-79ea-4701-b112-e7f801d2ed40" path="/var/lib/kubelet/pods/933f531c-79ea-4701-b112-e7f801d2ed40/volumes"
Sep 17 14:24:05.892803 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:05.892709    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:05.989581 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:05.989546    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:06.570876 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:06.570842    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:07.362666 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:07.362630    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:08.137372 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:08.137337    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:10.038263 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:10.038228    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:10.238496 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:10.238458    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:10.363156 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:10.363063    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:10.924838 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:10.924798    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:11.239884 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:11.239836    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:11.623948 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:11.623858    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:13.965472 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:13.965438    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:14.623035 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:14.623000    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:14.738689 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:14.738653    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:15.356215 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:15.356180    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:15.567000 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:15.566966    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:16.369010 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:16.368961    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:17.183249 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:17.183213    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:17.272658 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:17.272620    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf"]
Sep 17 14:24:17.272939 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:17.272912    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf" podUID="4802f1d6-92af-4711-ae3e-e85d0442a1ea" containerName="istio-proxy" containerID="cri-o://83fb30129881040c86f166e14c797dab7d1dcec817d17c57a21e050d51b5cefb" gracePeriod=30
Sep 17 14:24:17.308760 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:17.308717    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:17.402969 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:17.402926    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:17.775088 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:17.775057    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:17.949071 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:17.949023    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:18.028230 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.028132    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6"]
Sep 17 14:24:18.029216 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.029187    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f93aaa8-c6b7-404e-bb58-5b0786be570f" containerName="istio-proxy"
Sep 17 14:24:18.029372 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.029360    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f93aaa8-c6b7-404e-bb58-5b0786be570f" containerName="istio-proxy"
Sep 17 14:24:18.029486 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.029475    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f93aaa8-c6b7-404e-bb58-5b0786be570f" containerName="istio-proxy"
Sep 17 14:24:18.029568 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.029559    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f93aaa8-c6b7-404e-bb58-5b0786be570f" containerName="istio-proxy"
Sep 17 14:24:18.029669 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.029657    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="933f531c-79ea-4701-b112-e7f801d2ed40" containerName="revoke-keys"
Sep 17 14:24:18.029751 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.029742    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="933f531c-79ea-4701-b112-e7f801d2ed40" containerName="revoke-keys"
Sep 17 14:24:18.029952 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.029939    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f93aaa8-c6b7-404e-bb58-5b0786be570f" containerName="istio-proxy"
Sep 17 14:24:18.030044 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.030034    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="933f531c-79ea-4701-b112-e7f801d2ed40" containerName="revoke-keys"
Sep 17 14:24:18.030123 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.030114    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f93aaa8-c6b7-404e-bb58-5b0786be570f" containerName="istio-proxy"
Sep 17 14:24:18.035365 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.035338    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6"
Sep 17 14:24:18.037837 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.037816    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-0c1110-openshift-default-dockercfg-szdn7\""
Sep 17 14:24:18.044328 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.044298    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6"]
Sep 17 14:24:18.047739 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.047718    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:18.113842 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.113808    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43bc21dc-416c-42bd-8176-2cf465dd544e-workload-certs\") pod \"e2e-noten-0c1110-openshift-default-58659488d4-t86f6\" (UID: \"43bc21dc-416c-42bd-8176-2cf465dd544e\") " pod="openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6"
Sep 17 14:24:18.113842 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.113841    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43bc21dc-416c-42bd-8176-2cf465dd544e-istio-token\") pod \"e2e-noten-0c1110-openshift-default-58659488d4-t86f6\" (UID: \"43bc21dc-416c-42bd-8176-2cf465dd544e\") " pod="openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6"
Sep 17 14:24:18.114050 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.113869    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkj4s\" (UniqueName: \"kubernetes.io/projected/43bc21dc-416c-42bd-8176-2cf465dd544e-kube-api-access-fkj4s\") pod \"e2e-noten-0c1110-openshift-default-58659488d4-t86f6\" (UID: \"43bc21dc-416c-42bd-8176-2cf465dd544e\") " pod="openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6"
Sep 17 14:24:18.114050 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.113955    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43bc21dc-416c-42bd-8176-2cf465dd544e-credential-socket\") pod \"e2e-noten-0c1110-openshift-default-58659488d4-t86f6\" (UID: \"43bc21dc-416c-42bd-8176-2cf465dd544e\") " pod="openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6"
Sep 17 14:24:18.114050 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.113985    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43bc21dc-416c-42bd-8176-2cf465dd544e-istio-data\") pod \"e2e-noten-0c1110-openshift-default-58659488d4-t86f6\" (UID: \"43bc21dc-416c-42bd-8176-2cf465dd544e\") " pod="openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6"
Sep 17 14:24:18.114050 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.114012    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43bc21dc-416c-42bd-8176-2cf465dd544e-istiod-ca-cert\") pod \"e2e-noten-0c1110-openshift-default-58659488d4-t86f6\" (UID: \"43bc21dc-416c-42bd-8176-2cf465dd544e\") " pod="openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6"
Sep 17 14:24:18.114194 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.114059    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43bc21dc-416c-42bd-8176-2cf465dd544e-istio-envoy\") pod \"e2e-noten-0c1110-openshift-default-58659488d4-t86f6\" (UID: \"43bc21dc-416c-42bd-8176-2cf465dd544e\") " pod="openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6"
Sep 17 14:24:18.114194 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.114090    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43bc21dc-416c-42bd-8176-2cf465dd544e-istio-podinfo\") pod \"e2e-noten-0c1110-openshift-default-58659488d4-t86f6\" (UID: \"43bc21dc-416c-42bd-8176-2cf465dd544e\") " pod="openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6"
Sep 17 14:24:18.114194 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.114123    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43bc21dc-416c-42bd-8176-2cf465dd544e-workload-socket\") pod \"e2e-noten-0c1110-openshift-default-58659488d4-t86f6\" (UID: \"43bc21dc-416c-42bd-8176-2cf465dd544e\") " pod="openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6"
Sep 17 14:24:18.119855 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.119809    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:18.215427 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.215387    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43bc21dc-416c-42bd-8176-2cf465dd544e-istiod-ca-cert\") pod \"e2e-noten-0c1110-openshift-default-58659488d4-t86f6\" (UID: \"43bc21dc-416c-42bd-8176-2cf465dd544e\") " pod="openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6"
Sep 17 14:24:18.215713 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.215458    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43bc21dc-416c-42bd-8176-2cf465dd544e-istio-envoy\") pod \"e2e-noten-0c1110-openshift-default-58659488d4-t86f6\" (UID: \"43bc21dc-416c-42bd-8176-2cf465dd544e\") " pod="openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6"
Sep 17 14:24:18.215713 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.215572    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43bc21dc-416c-42bd-8176-2cf465dd544e-istio-podinfo\") pod \"e2e-noten-0c1110-openshift-default-58659488d4-t86f6\" (UID: \"43bc21dc-416c-42bd-8176-2cf465dd544e\") " pod="openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6"
Sep 17 14:24:18.215713 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.215634    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43bc21dc-416c-42bd-8176-2cf465dd544e-workload-socket\") pod \"e2e-noten-0c1110-openshift-default-58659488d4-t86f6\" (UID: \"43bc21dc-416c-42bd-8176-2cf465dd544e\") " pod="openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6"
Sep 17 14:24:18.215913 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.215762    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43bc21dc-416c-42bd-8176-2cf465dd544e-workload-certs\") pod \"e2e-noten-0c1110-openshift-default-58659488d4-t86f6\" (UID: \"43bc21dc-416c-42bd-8176-2cf465dd544e\") " pod="openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6"
Sep 17 14:24:18.215913 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.215792    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43bc21dc-416c-42bd-8176-2cf465dd544e-istio-token\") pod \"e2e-noten-0c1110-openshift-default-58659488d4-t86f6\" (UID: \"43bc21dc-416c-42bd-8176-2cf465dd544e\") " pod="openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6"
Sep 17 14:24:18.215913 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.215837    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkj4s\" (UniqueName: \"kubernetes.io/projected/43bc21dc-416c-42bd-8176-2cf465dd544e-kube-api-access-fkj4s\") pod \"e2e-noten-0c1110-openshift-default-58659488d4-t86f6\" (UID: \"43bc21dc-416c-42bd-8176-2cf465dd544e\") " pod="openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6"
Sep 17 14:24:18.215913 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.215878    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43bc21dc-416c-42bd-8176-2cf465dd544e-credential-socket\") pod \"e2e-noten-0c1110-openshift-default-58659488d4-t86f6\" (UID: \"43bc21dc-416c-42bd-8176-2cf465dd544e\") " pod="openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6"
Sep 17 14:24:18.215913 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.215906    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43bc21dc-416c-42bd-8176-2cf465dd544e-istio-data\") pod \"e2e-noten-0c1110-openshift-default-58659488d4-t86f6\" (UID: \"43bc21dc-416c-42bd-8176-2cf465dd544e\") " pod="openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6"
Sep 17 14:24:18.216561 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.216503    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43bc21dc-416c-42bd-8176-2cf465dd544e-istiod-ca-cert\") pod \"e2e-noten-0c1110-openshift-default-58659488d4-t86f6\" (UID: \"43bc21dc-416c-42bd-8176-2cf465dd544e\") " pod="openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6"
Sep 17 14:24:18.216699 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.216522    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43bc21dc-416c-42bd-8176-2cf465dd544e-workload-certs\") pod \"e2e-noten-0c1110-openshift-default-58659488d4-t86f6\" (UID: \"43bc21dc-416c-42bd-8176-2cf465dd544e\") " pod="openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6"
Sep 17 14:24:18.219052 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.218965    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43bc21dc-416c-42bd-8176-2cf465dd544e-credential-socket\") pod \"e2e-noten-0c1110-openshift-default-58659488d4-t86f6\" (UID: \"43bc21dc-416c-42bd-8176-2cf465dd544e\") " pod="openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6"
Sep 17 14:24:18.219052 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.219017    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43bc21dc-416c-42bd-8176-2cf465dd544e-workload-socket\") pod \"e2e-noten-0c1110-openshift-default-58659488d4-t86f6\" (UID: \"43bc21dc-416c-42bd-8176-2cf465dd544e\") " pod="openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6"
Sep 17 14:24:18.219236 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.219106    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43bc21dc-416c-42bd-8176-2cf465dd544e-istio-podinfo\") pod \"e2e-noten-0c1110-openshift-default-58659488d4-t86f6\" (UID: \"43bc21dc-416c-42bd-8176-2cf465dd544e\") " pod="openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6"
Sep 17 14:24:18.219556 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.219537    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43bc21dc-416c-42bd-8176-2cf465dd544e-istio-data\") pod \"e2e-noten-0c1110-openshift-default-58659488d4-t86f6\" (UID: \"43bc21dc-416c-42bd-8176-2cf465dd544e\") " pod="openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6"
Sep 17 14:24:18.219911 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.219852    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43bc21dc-416c-42bd-8176-2cf465dd544e-istio-envoy\") pod \"e2e-noten-0c1110-openshift-default-58659488d4-t86f6\" (UID: \"43bc21dc-416c-42bd-8176-2cf465dd544e\") " pod="openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6"
Sep 17 14:24:18.230713 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.228274    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43bc21dc-416c-42bd-8176-2cf465dd544e-istio-token\") pod \"e2e-noten-0c1110-openshift-default-58659488d4-t86f6\" (UID: \"43bc21dc-416c-42bd-8176-2cf465dd544e\") " pod="openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6"
Sep 17 14:24:18.230713 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.230281    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:18.230713 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.230493    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkj4s\" (UniqueName: \"kubernetes.io/projected/43bc21dc-416c-42bd-8176-2cf465dd544e-kube-api-access-fkj4s\") pod \"e2e-noten-0c1110-openshift-default-58659488d4-t86f6\" (UID: \"43bc21dc-416c-42bd-8176-2cf465dd544e\") " pod="openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6"
Sep 17 14:24:18.347764 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.347670    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6"
Sep 17 14:24:18.499415 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.499384    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:18.609424 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.609348    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:18.632018 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.631985    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6"]
Sep 17 14:24:18.634183 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:24:18.634151    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43bc21dc_416c_42bd_8176_2cf465dd544e.slice/crio-99b6e96335c9d2da6e39ce69ebf9f183094864fc880449fbdcd14fddc37960d5 WatchSource:0}: Error finding container 99b6e96335c9d2da6e39ce69ebf9f183094864fc880449fbdcd14fddc37960d5: Status 404 returned error can't find the container with id 99b6e96335c9d2da6e39ce69ebf9f183094864fc880449fbdcd14fddc37960d5
Sep 17 14:24:18.637069 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.637030    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:24:18.637167 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.637118    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:24:18.637167 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:18.637160    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:24:19.008586 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:19.008549    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6" event={"ID":"43bc21dc-416c-42bd-8176-2cf465dd544e","Type":"ContainerStarted","Data":"b8646fd9c22472cb05d857e49e036cd5e3e6e915d13a30952cd737c434b8bd9b"}
Sep 17 14:24:19.008835 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:19.008629    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6" event={"ID":"43bc21dc-416c-42bd-8176-2cf465dd544e","Type":"ContainerStarted","Data":"99b6e96335c9d2da6e39ce69ebf9f183094864fc880449fbdcd14fddc37960d5"}
Sep 17 14:24:19.034279 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:19.034212    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6" podStartSLOduration=1.034187148 podStartE2EDuration="1.034187148s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:24:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:24:19.030367329 +0000 UTC m=+1160.031910802" watchObservedRunningTime="2026-09-17 14:24:19.034187148 +0000 UTC m=+1160.035730623"
Sep 17 14:24:19.348230 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:19.348144    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6"
Sep 17 14:24:19.353739 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:19.353712    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6"
Sep 17 14:24:19.529628 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:19.529574    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:19.630250 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:19.630066    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:19.975355 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:19.975294    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:20.015440 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:20.015407    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6"
Sep 17 14:24:20.016494 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:20.016468    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6"
Sep 17 14:24:22.547771 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:22.547702    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf"
Sep 17 14:24:22.557685 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:22.557556    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:22.674725 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:22.674693    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4802f1d6-92af-4711-ae3e-e85d0442a1ea-istio-podinfo\") pod \"4802f1d6-92af-4711-ae3e-e85d0442a1ea\" (UID: \"4802f1d6-92af-4711-ae3e-e85d0442a1ea\") "
Sep 17 14:24:22.674725 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:22.674724    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4802f1d6-92af-4711-ae3e-e85d0442a1ea-istio-token\") pod \"4802f1d6-92af-4711-ae3e-e85d0442a1ea\" (UID: \"4802f1d6-92af-4711-ae3e-e85d0442a1ea\") "
Sep 17 14:24:22.674984 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:22.674762    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4802f1d6-92af-4711-ae3e-e85d0442a1ea-istio-envoy\") pod \"4802f1d6-92af-4711-ae3e-e85d0442a1ea\" (UID: \"4802f1d6-92af-4711-ae3e-e85d0442a1ea\") "
Sep 17 14:24:22.674984 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:22.674779    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4802f1d6-92af-4711-ae3e-e85d0442a1ea-workload-socket\") pod \"4802f1d6-92af-4711-ae3e-e85d0442a1ea\" (UID: \"4802f1d6-92af-4711-ae3e-e85d0442a1ea\") "
Sep 17 14:24:22.674984 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:22.674861    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4802f1d6-92af-4711-ae3e-e85d0442a1ea-istiod-ca-cert\") pod \"4802f1d6-92af-4711-ae3e-e85d0442a1ea\" (UID: \"4802f1d6-92af-4711-ae3e-e85d0442a1ea\") "
Sep 17 14:24:22.674984 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:22.674924    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4802f1d6-92af-4711-ae3e-e85d0442a1ea-istio-data\") pod \"4802f1d6-92af-4711-ae3e-e85d0442a1ea\" (UID: \"4802f1d6-92af-4711-ae3e-e85d0442a1ea\") "
Sep 17 14:24:22.674984 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:22.674971    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4802f1d6-92af-4711-ae3e-e85d0442a1ea-workload-certs\") pod \"4802f1d6-92af-4711-ae3e-e85d0442a1ea\" (UID: \"4802f1d6-92af-4711-ae3e-e85d0442a1ea\") "
Sep 17 14:24:22.675250 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:22.675034    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4802f1d6-92af-4711-ae3e-e85d0442a1ea-credential-socket\") pod \"4802f1d6-92af-4711-ae3e-e85d0442a1ea\" (UID: \"4802f1d6-92af-4711-ae3e-e85d0442a1ea\") "
Sep 17 14:24:22.675250 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:22.675059    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lpvtv\" (UniqueName: \"kubernetes.io/projected/4802f1d6-92af-4711-ae3e-e85d0442a1ea-kube-api-access-lpvtv\") pod \"4802f1d6-92af-4711-ae3e-e85d0442a1ea\" (UID: \"4802f1d6-92af-4711-ae3e-e85d0442a1ea\") "
Sep 17 14:24:22.675250 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:22.675164    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4802f1d6-92af-4711-ae3e-e85d0442a1ea-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4802f1d6-92af-4711-ae3e-e85d0442a1ea" (UID: "4802f1d6-92af-4711-ae3e-e85d0442a1ea"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:24:22.675455 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:22.675247    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4802f1d6-92af-4711-ae3e-e85d0442a1ea-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4802f1d6-92af-4711-ae3e-e85d0442a1ea" (UID: "4802f1d6-92af-4711-ae3e-e85d0442a1ea"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:24:22.675455 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:22.675310    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4802f1d6-92af-4711-ae3e-e85d0442a1ea-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4802f1d6-92af-4711-ae3e-e85d0442a1ea" (UID: "4802f1d6-92af-4711-ae3e-e85d0442a1ea"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:24:22.675455 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:22.675159    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4802f1d6-92af-4711-ae3e-e85d0442a1ea-istio-data" (OuterVolumeSpecName: "istio-data") pod "4802f1d6-92af-4711-ae3e-e85d0442a1ea" (UID: "4802f1d6-92af-4711-ae3e-e85d0442a1ea"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:24:22.675455 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:22.675404    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4802f1d6-92af-4711-ae3e-e85d0442a1ea-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4802f1d6-92af-4711-ae3e-e85d0442a1ea" (UID: "4802f1d6-92af-4711-ae3e-e85d0442a1ea"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:24:22.675723 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:22.675529    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4802f1d6-92af-4711-ae3e-e85d0442a1ea-istiod-ca-cert\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:24:22.675723 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:22.675546    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4802f1d6-92af-4711-ae3e-e85d0442a1ea-istio-data\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:24:22.675723 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:22.675559    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4802f1d6-92af-4711-ae3e-e85d0442a1ea-workload-certs\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:24:22.675723 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:22.675569    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4802f1d6-92af-4711-ae3e-e85d0442a1ea-credential-socket\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:24:22.675723 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:22.675583    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4802f1d6-92af-4711-ae3e-e85d0442a1ea-workload-socket\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:24:22.677117 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:22.677091    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4802f1d6-92af-4711-ae3e-e85d0442a1ea-kube-api-access-lpvtv" (OuterVolumeSpecName: "kube-api-access-lpvtv") pod "4802f1d6-92af-4711-ae3e-e85d0442a1ea" (UID: "4802f1d6-92af-4711-ae3e-e85d0442a1ea"). InnerVolumeSpecName "kube-api-access-lpvtv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:24:22.677240 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:22.677191    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4802f1d6-92af-4711-ae3e-e85d0442a1ea-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4802f1d6-92af-4711-ae3e-e85d0442a1ea" (UID: "4802f1d6-92af-4711-ae3e-e85d0442a1ea"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:24:22.677240 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:22.677224    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4802f1d6-92af-4711-ae3e-e85d0442a1ea-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4802f1d6-92af-4711-ae3e-e85d0442a1ea" (UID: "4802f1d6-92af-4711-ae3e-e85d0442a1ea"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 14:24:22.677326 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:22.677249    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4802f1d6-92af-4711-ae3e-e85d0442a1ea-istio-token" (OuterVolumeSpecName: "istio-token") pod "4802f1d6-92af-4711-ae3e-e85d0442a1ea" (UID: "4802f1d6-92af-4711-ae3e-e85d0442a1ea"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:24:22.776482 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:22.776442    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4802f1d6-92af-4711-ae3e-e85d0442a1ea-istio-podinfo\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:24:22.776482 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:22.776474    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4802f1d6-92af-4711-ae3e-e85d0442a1ea-istio-token\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:24:22.776482 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:22.776489    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4802f1d6-92af-4711-ae3e-e85d0442a1ea-istio-envoy\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:24:22.776864 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:22.776501    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lpvtv\" (UniqueName: \"kubernetes.io/projected/4802f1d6-92af-4711-ae3e-e85d0442a1ea-kube-api-access-lpvtv\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:24:23.037577 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:23.037452    2582 generic.go:356] "Generic (PLEG): container finished" podID="4802f1d6-92af-4711-ae3e-e85d0442a1ea" containerID="83fb30129881040c86f166e14c797dab7d1dcec817d17c57a21e050d51b5cefb" exitCode=0
Sep 17 14:24:23.037577 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:23.037495    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf"
Sep 17 14:24:23.037577 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:23.037508    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf" event={"ID":"4802f1d6-92af-4711-ae3e-e85d0442a1ea","Type":"ContainerDied","Data":"83fb30129881040c86f166e14c797dab7d1dcec817d17c57a21e050d51b5cefb"}
Sep 17 14:24:23.037577 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:23.037564    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf" event={"ID":"4802f1d6-92af-4711-ae3e-e85d0442a1ea","Type":"ContainerDied","Data":"4c9976f2c1f268399f98a60d03dbb1a523a15d6a746e00bbc3e317c95ef80ef7"}
Sep 17 14:24:23.037577 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:23.037586    2582 scope.go:117] "RemoveContainer" containerID="83fb30129881040c86f166e14c797dab7d1dcec817d17c57a21e050d51b5cefb"
Sep 17 14:24:23.048314 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:23.048294    2582 scope.go:117] "RemoveContainer" containerID="c3cd5ebd47ac07b15b2a05767a0dfb0dacdd6f48b9e838b206bbeb6248eb0fcb"
Sep 17 14:24:23.057249 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:23.057229    2582 scope.go:117] "RemoveContainer" containerID="83fb30129881040c86f166e14c797dab7d1dcec817d17c57a21e050d51b5cefb"
Sep 17 14:24:23.057636 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:24:23.057522    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83fb30129881040c86f166e14c797dab7d1dcec817d17c57a21e050d51b5cefb\": container with ID starting with 83fb30129881040c86f166e14c797dab7d1dcec817d17c57a21e050d51b5cefb not found: ID does not exist" containerID="83fb30129881040c86f166e14c797dab7d1dcec817d17c57a21e050d51b5cefb"
Sep 17 14:24:23.057636 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:23.057560    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83fb30129881040c86f166e14c797dab7d1dcec817d17c57a21e050d51b5cefb"} err="failed to get container status \"83fb30129881040c86f166e14c797dab7d1dcec817d17c57a21e050d51b5cefb\": rpc error: code = NotFound desc = could not find container \"83fb30129881040c86f166e14c797dab7d1dcec817d17c57a21e050d51b5cefb\": container with ID starting with 83fb30129881040c86f166e14c797dab7d1dcec817d17c57a21e050d51b5cefb not found: ID does not exist"
Sep 17 14:24:23.057636 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:23.057583    2582 scope.go:117] "RemoveContainer" containerID="c3cd5ebd47ac07b15b2a05767a0dfb0dacdd6f48b9e838b206bbeb6248eb0fcb"
Sep 17 14:24:23.058124 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:24:23.058090    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3cd5ebd47ac07b15b2a05767a0dfb0dacdd6f48b9e838b206bbeb6248eb0fcb\": container with ID starting with c3cd5ebd47ac07b15b2a05767a0dfb0dacdd6f48b9e838b206bbeb6248eb0fcb not found: ID does not exist" containerID="c3cd5ebd47ac07b15b2a05767a0dfb0dacdd6f48b9e838b206bbeb6248eb0fcb"
Sep 17 14:24:23.058257 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:23.058134    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3cd5ebd47ac07b15b2a05767a0dfb0dacdd6f48b9e838b206bbeb6248eb0fcb"} err="failed to get container status \"c3cd5ebd47ac07b15b2a05767a0dfb0dacdd6f48b9e838b206bbeb6248eb0fcb\": rpc error: code = NotFound desc = could not find container \"c3cd5ebd47ac07b15b2a05767a0dfb0dacdd6f48b9e838b206bbeb6248eb0fcb\": container with ID starting with c3cd5ebd47ac07b15b2a05767a0dfb0dacdd6f48b9e838b206bbeb6248eb0fcb not found: ID does not exist"
Sep 17 14:24:23.060988 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:23.060968    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf"]
Sep 17 14:24:23.066806 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:23.066786    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-68183c-openshift-default-56b46bfb7b-r4rdf"]
Sep 17 14:24:23.294976 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:23.294890    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:23.493953 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:23.493922    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4802f1d6-92af-4711-ae3e-e85d0442a1ea" path="/var/lib/kubelet/pods/4802f1d6-92af-4711-ae3e-e85d0442a1ea/volumes"
Sep 17 14:24:38.974014 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:38.973975    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:40.284236 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:40.284194    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:41.063157 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:41.063125    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:41.699894 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:41.699858    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:42.416785 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:42.416742    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:43.111599 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:43.111565    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:43.733393 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:43.733353    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:44.580852 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:44.580816    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:44.710559 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:44.710523    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:45.058271 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:45.058239    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:45.385326 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:45.385248    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr"]
Sep 17 14:24:45.385737 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:45.385724    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4802f1d6-92af-4711-ae3e-e85d0442a1ea" containerName="istio-proxy"
Sep 17 14:24:45.385789 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:45.385739    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4802f1d6-92af-4711-ae3e-e85d0442a1ea" containerName="istio-proxy"
Sep 17 14:24:45.385789 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:45.385754    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4802f1d6-92af-4711-ae3e-e85d0442a1ea" containerName="istio-proxy"
Sep 17 14:24:45.385789 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:45.385759    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4802f1d6-92af-4711-ae3e-e85d0442a1ea" containerName="istio-proxy"
Sep 17 14:24:45.385892 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:45.385833    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4802f1d6-92af-4711-ae3e-e85d0442a1ea" containerName="istio-proxy"
Sep 17 14:24:45.385892 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:45.385842    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4802f1d6-92af-4711-ae3e-e85d0442a1ea" containerName="istio-proxy"
Sep 17 14:24:45.389181 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:45.389166    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr"
Sep 17 14:24:45.391533 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:45.391513    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-b83f70-openshift-default-dockercfg-x5g6r\""
Sep 17 14:24:45.391981 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:45.391950    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:45.402037 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:45.402012    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr"]
Sep 17 14:24:45.511304 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:45.511274    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/932dad53-ff0a-412d-a874-55e0edc95702-istio-podinfo\") pod \"e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr\" (UID: \"932dad53-ff0a-412d-a874-55e0edc95702\") " pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr"
Sep 17 14:24:45.511480 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:45.511315    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hnq7\" (UniqueName: \"kubernetes.io/projected/932dad53-ff0a-412d-a874-55e0edc95702-kube-api-access-9hnq7\") pod \"e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr\" (UID: \"932dad53-ff0a-412d-a874-55e0edc95702\") " pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr"
Sep 17 14:24:45.511480 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:45.511345    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/932dad53-ff0a-412d-a874-55e0edc95702-istiod-ca-cert\") pod \"e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr\" (UID: \"932dad53-ff0a-412d-a874-55e0edc95702\") " pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr"
Sep 17 14:24:45.511480 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:45.511422    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/932dad53-ff0a-412d-a874-55e0edc95702-workload-certs\") pod \"e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr\" (UID: \"932dad53-ff0a-412d-a874-55e0edc95702\") " pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr"
Sep 17 14:24:45.511480 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:45.511466    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/932dad53-ff0a-412d-a874-55e0edc95702-istio-data\") pod \"e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr\" (UID: \"932dad53-ff0a-412d-a874-55e0edc95702\") " pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr"
Sep 17 14:24:45.511706 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:45.511557    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/932dad53-ff0a-412d-a874-55e0edc95702-istio-envoy\") pod \"e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr\" (UID: \"932dad53-ff0a-412d-a874-55e0edc95702\") " pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr"
Sep 17 14:24:45.511706 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:45.511627    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/932dad53-ff0a-412d-a874-55e0edc95702-workload-socket\") pod \"e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr\" (UID: \"932dad53-ff0a-412d-a874-55e0edc95702\") " pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr"
Sep 17 14:24:45.511706 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:45.511658    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/932dad53-ff0a-412d-a874-55e0edc95702-istio-token\") pod \"e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr\" (UID: \"932dad53-ff0a-412d-a874-55e0edc95702\") " pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr"
Sep 17 14:24:45.511706 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:45.511693    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/932dad53-ff0a-412d-a874-55e0edc95702-credential-socket\") pod \"e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr\" (UID: \"932dad53-ff0a-412d-a874-55e0edc95702\") " pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr"
Sep 17 14:24:45.612988 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:45.612957    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hnq7\" (UniqueName: \"kubernetes.io/projected/932dad53-ff0a-412d-a874-55e0edc95702-kube-api-access-9hnq7\") pod \"e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr\" (UID: \"932dad53-ff0a-412d-a874-55e0edc95702\") " pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr"
Sep 17 14:24:45.613427 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:45.613001    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/932dad53-ff0a-412d-a874-55e0edc95702-istiod-ca-cert\") pod \"e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr\" (UID: \"932dad53-ff0a-412d-a874-55e0edc95702\") " pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr"
Sep 17 14:24:45.613427 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:45.613142    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/932dad53-ff0a-412d-a874-55e0edc95702-workload-certs\") pod \"e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr\" (UID: \"932dad53-ff0a-412d-a874-55e0edc95702\") " pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr"
Sep 17 14:24:45.613427 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:45.613188    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/932dad53-ff0a-412d-a874-55e0edc95702-istio-data\") pod \"e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr\" (UID: \"932dad53-ff0a-412d-a874-55e0edc95702\") " pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr"
Sep 17 14:24:45.613427 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:45.613323    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/932dad53-ff0a-412d-a874-55e0edc95702-istio-envoy\") pod \"e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr\" (UID: \"932dad53-ff0a-412d-a874-55e0edc95702\") " pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr"
Sep 17 14:24:45.613691 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:45.613465    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/932dad53-ff0a-412d-a874-55e0edc95702-workload-socket\") pod \"e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr\" (UID: \"932dad53-ff0a-412d-a874-55e0edc95702\") " pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr"
Sep 17 14:24:45.613691 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:45.613478    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/932dad53-ff0a-412d-a874-55e0edc95702-workload-certs\") pod \"e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr\" (UID: \"932dad53-ff0a-412d-a874-55e0edc95702\") " pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr"
Sep 17 14:24:45.613691 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:45.613499    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/932dad53-ff0a-412d-a874-55e0edc95702-istio-token\") pod \"e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr\" (UID: \"932dad53-ff0a-412d-a874-55e0edc95702\") " pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr"
Sep 17 14:24:45.613691 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:45.613597    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/932dad53-ff0a-412d-a874-55e0edc95702-credential-socket\") pod \"e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr\" (UID: \"932dad53-ff0a-412d-a874-55e0edc95702\") " pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr"
Sep 17 14:24:45.613691 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:45.613668    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/932dad53-ff0a-412d-a874-55e0edc95702-istio-podinfo\") pod \"e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr\" (UID: \"932dad53-ff0a-412d-a874-55e0edc95702\") " pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr"
Sep 17 14:24:45.614178 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:45.614151    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/932dad53-ff0a-412d-a874-55e0edc95702-istio-data\") pod \"e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr\" (UID: \"932dad53-ff0a-412d-a874-55e0edc95702\") " pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr"
Sep 17 14:24:45.614309 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:45.614266    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/932dad53-ff0a-412d-a874-55e0edc95702-workload-socket\") pod \"e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr\" (UID: \"932dad53-ff0a-412d-a874-55e0edc95702\") " pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr"
Sep 17 14:24:45.614309 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:45.614283    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/932dad53-ff0a-412d-a874-55e0edc95702-credential-socket\") pod \"e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr\" (UID: \"932dad53-ff0a-412d-a874-55e0edc95702\") " pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr"
Sep 17 14:24:45.614549 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:45.614524    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/932dad53-ff0a-412d-a874-55e0edc95702-istiod-ca-cert\") pod \"e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr\" (UID: \"932dad53-ff0a-412d-a874-55e0edc95702\") " pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr"
Sep 17 14:24:45.616139 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:45.616119    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/932dad53-ff0a-412d-a874-55e0edc95702-istio-envoy\") pod \"e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr\" (UID: \"932dad53-ff0a-412d-a874-55e0edc95702\") " pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr"
Sep 17 14:24:45.616360 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:45.616336    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/932dad53-ff0a-412d-a874-55e0edc95702-istio-podinfo\") pod \"e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr\" (UID: \"932dad53-ff0a-412d-a874-55e0edc95702\") " pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr"
Sep 17 14:24:45.621427 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:45.621405    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/932dad53-ff0a-412d-a874-55e0edc95702-istio-token\") pod \"e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr\" (UID: \"932dad53-ff0a-412d-a874-55e0edc95702\") " pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr"
Sep 17 14:24:45.621819 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:45.621803    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hnq7\" (UniqueName: \"kubernetes.io/projected/932dad53-ff0a-412d-a874-55e0edc95702-kube-api-access-9hnq7\") pod \"e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr\" (UID: \"932dad53-ff0a-412d-a874-55e0edc95702\") " pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr"
Sep 17 14:24:45.699693 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:45.699661    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr"
Sep 17 14:24:45.842776 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:45.842742    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:45.988719 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:45.988684    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr"]
Sep 17 14:24:45.990342 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:24:45.990317    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod932dad53_ff0a_412d_a874_55e0edc95702.slice/crio-a7223bbbcf1b0a927e3f69ed9130efbbaae18fc6898640f3dc9d147c8c4392bc WatchSource:0}: Error finding container a7223bbbcf1b0a927e3f69ed9130efbbaae18fc6898640f3dc9d147c8c4392bc: Status 404 returned error can't find the container with id a7223bbbcf1b0a927e3f69ed9130efbbaae18fc6898640f3dc9d147c8c4392bc
Sep 17 14:24:45.992702 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:45.992673    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:24:45.992783 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:45.992736    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:24:45.992783 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:45.992766    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:24:46.191583 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:46.191539    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr" event={"ID":"932dad53-ff0a-412d-a874-55e0edc95702","Type":"ContainerStarted","Data":"d62d9780538b39ced2431aa400c5c9dd51a0c22d35662bbaa79ac53a819a1117"}
Sep 17 14:24:46.191807 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:46.191620    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr" event={"ID":"932dad53-ff0a-412d-a874-55e0edc95702","Type":"ContainerStarted","Data":"a7223bbbcf1b0a927e3f69ed9130efbbaae18fc6898640f3dc9d147c8c4392bc"}
Sep 17 14:24:46.214047 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:46.213996    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr" podStartSLOduration=1.213982816 podStartE2EDuration="1.213982816s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:24:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:24:46.212079721 +0000 UTC m=+1187.213623229" watchObservedRunningTime="2026-09-17 14:24:46.213982816 +0000 UTC m=+1187.215526289"
Sep 17 14:24:46.700422 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:46.700389    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr"
Sep 17 14:24:47.611665 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:47.611600    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:47.706266 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:47.706235    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr"
Sep 17 14:24:48.203984 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:48.203953    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr"
Sep 17 14:24:48.204980 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:48.204956    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr"
Sep 17 14:24:48.218007 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:48.217977    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:48.443931 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:48.443891    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:49.063929 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:49.063889    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:49.850225 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:49.850195    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:50.558062 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:50.558031    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:51.454480 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:51.454448    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:52.049422 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:52.049304    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:52.164676 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:52.164641    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:52.560242 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:52.560204    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:52.658119 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:52.658088    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:52.758440 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:52.758409    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:24:59.526722 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:59.526684    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-rr97k_45ffa13d-635a-46e8-8894-30902b9cf719/console-operator/2.log"
Sep 17 14:24:59.527947 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:24:59.527928    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-rr97k_45ffa13d-635a-46e8-8894-30902b9cf719/console-operator/2.log"
Sep 17 14:25:09.579794 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:09.579754    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46"]
Sep 17 14:25:09.586771 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:09.586740    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46"
Sep 17 14:25:09.590984 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:09.590935    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-fcd9f3\"/\"e2e-conflict-a-9497140e-kserve-self-signed-certs\""
Sep 17 14:25:09.591349 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:09.591327    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-fcd9f3\"/\"default-dockercfg-9sc5l\""
Sep 17 14:25:09.591514 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:09.591489    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-fcd9f3\"/\"openshift-service-ca.crt\""
Sep 17 14:25:09.591669 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:09.591649    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-fcd9f3\"/\"kube-root-ca.crt\""
Sep 17 14:25:09.595248 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:09.595215    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46"]
Sep 17 14:25:09.680681 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:09.680602    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f20fa859-9ed0-4465-ab5c-37adbcbf12a9-tls-certs\") pod \"e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46\" (UID: \"f20fa859-9ed0-4465-ab5c-37adbcbf12a9\") " pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46"
Sep 17 14:25:09.680903 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:09.680729    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7s69\" (UniqueName: \"kubernetes.io/projected/f20fa859-9ed0-4465-ab5c-37adbcbf12a9-kube-api-access-x7s69\") pod \"e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46\" (UID: \"f20fa859-9ed0-4465-ab5c-37adbcbf12a9\") " pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46"
Sep 17 14:25:09.680903 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:09.680768    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f20fa859-9ed0-4465-ab5c-37adbcbf12a9-dshm\") pod \"e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46\" (UID: \"f20fa859-9ed0-4465-ab5c-37adbcbf12a9\") " pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46"
Sep 17 14:25:09.680903 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:09.680851    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f20fa859-9ed0-4465-ab5c-37adbcbf12a9-model-cache\") pod \"e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46\" (UID: \"f20fa859-9ed0-4465-ab5c-37adbcbf12a9\") " pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46"
Sep 17 14:25:09.681026 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:09.680976    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f20fa859-9ed0-4465-ab5c-37adbcbf12a9-home\") pod \"e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46\" (UID: \"f20fa859-9ed0-4465-ab5c-37adbcbf12a9\") " pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46"
Sep 17 14:25:09.681075 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:09.681034    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f20fa859-9ed0-4465-ab5c-37adbcbf12a9-tmp-dir\") pod \"e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46\" (UID: \"f20fa859-9ed0-4465-ab5c-37adbcbf12a9\") " pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46"
Sep 17 14:25:09.782621 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:09.782569    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f20fa859-9ed0-4465-ab5c-37adbcbf12a9-model-cache\") pod \"e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46\" (UID: \"f20fa859-9ed0-4465-ab5c-37adbcbf12a9\") " pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46"
Sep 17 14:25:09.782816 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:09.782679    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f20fa859-9ed0-4465-ab5c-37adbcbf12a9-home\") pod \"e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46\" (UID: \"f20fa859-9ed0-4465-ab5c-37adbcbf12a9\") " pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46"
Sep 17 14:25:09.782816 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:09.782718    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f20fa859-9ed0-4465-ab5c-37adbcbf12a9-tmp-dir\") pod \"e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46\" (UID: \"f20fa859-9ed0-4465-ab5c-37adbcbf12a9\") " pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46"
Sep 17 14:25:09.782816 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:09.782740    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f20fa859-9ed0-4465-ab5c-37adbcbf12a9-tls-certs\") pod \"e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46\" (UID: \"f20fa859-9ed0-4465-ab5c-37adbcbf12a9\") " pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46"
Sep 17 14:25:09.782816 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:09.782771    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7s69\" (UniqueName: \"kubernetes.io/projected/f20fa859-9ed0-4465-ab5c-37adbcbf12a9-kube-api-access-x7s69\") pod \"e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46\" (UID: \"f20fa859-9ed0-4465-ab5c-37adbcbf12a9\") " pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46"
Sep 17 14:25:09.782816 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:09.782802    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f20fa859-9ed0-4465-ab5c-37adbcbf12a9-dshm\") pod \"e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46\" (UID: \"f20fa859-9ed0-4465-ab5c-37adbcbf12a9\") " pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46"
Sep 17 14:25:09.783092 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:09.783061    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f20fa859-9ed0-4465-ab5c-37adbcbf12a9-model-cache\") pod \"e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46\" (UID: \"f20fa859-9ed0-4465-ab5c-37adbcbf12a9\") " pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46"
Sep 17 14:25:09.783150 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:09.783093    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f20fa859-9ed0-4465-ab5c-37adbcbf12a9-home\") pod \"e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46\" (UID: \"f20fa859-9ed0-4465-ab5c-37adbcbf12a9\") " pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46"
Sep 17 14:25:09.783216 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:09.783194    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f20fa859-9ed0-4465-ab5c-37adbcbf12a9-tmp-dir\") pod \"e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46\" (UID: \"f20fa859-9ed0-4465-ab5c-37adbcbf12a9\") " pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46"
Sep 17 14:25:09.785079 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:09.785050    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f20fa859-9ed0-4465-ab5c-37adbcbf12a9-dshm\") pod \"e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46\" (UID: \"f20fa859-9ed0-4465-ab5c-37adbcbf12a9\") " pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46"
Sep 17 14:25:09.785362 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:09.785341    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f20fa859-9ed0-4465-ab5c-37adbcbf12a9-tls-certs\") pod \"e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46\" (UID: \"f20fa859-9ed0-4465-ab5c-37adbcbf12a9\") " pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46"
Sep 17 14:25:09.791824 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:09.791802    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7s69\" (UniqueName: \"kubernetes.io/projected/f20fa859-9ed0-4465-ab5c-37adbcbf12a9-kube-api-access-x7s69\") pod \"e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46\" (UID: \"f20fa859-9ed0-4465-ab5c-37adbcbf12a9\") " pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46"
Sep 17 14:25:09.901197 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:09.901102    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46"
Sep 17 14:25:10.042346 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:10.042299    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:25:10.191944 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:10.191907    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46"]
Sep 17 14:25:10.193447 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:25:10.193414    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf20fa859_9ed0_4465_ab5c_37adbcbf12a9.slice/crio-ba07637042f10f5c13d0d71430251677787b04042d7fd8c3e850d44abe405fc2 WatchSource:0}: Error finding container ba07637042f10f5c13d0d71430251677787b04042d7fd8c3e850d44abe405fc2: Status 404 returned error can't find the container with id ba07637042f10f5c13d0d71430251677787b04042d7fd8c3e850d44abe405fc2
Sep 17 14:25:10.345815 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:10.345784    2582 generic.go:356] "Generic (PLEG): container finished" podID="932dad53-ff0a-412d-a874-55e0edc95702" containerID="d62d9780538b39ced2431aa400c5c9dd51a0c22d35662bbaa79ac53a819a1117" exitCode=0
Sep 17 14:25:10.346000 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:10.345860    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr" event={"ID":"932dad53-ff0a-412d-a874-55e0edc95702","Type":"ContainerDied","Data":"d62d9780538b39ced2431aa400c5c9dd51a0c22d35662bbaa79ac53a819a1117"}
Sep 17 14:25:10.346427 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:10.346403    2582 scope.go:117] "RemoveContainer" containerID="d62d9780538b39ced2431aa400c5c9dd51a0c22d35662bbaa79ac53a819a1117"
Sep 17 14:25:10.347660 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:10.347638    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46" event={"ID":"f20fa859-9ed0-4465-ab5c-37adbcbf12a9","Type":"ContainerStarted","Data":"c196079f64c3ca0c9604f90269fc3117618d7875000bedf5e134129658feff42"}
Sep 17 14:25:10.347755 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:10.347672    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46" event={"ID":"f20fa859-9ed0-4465-ab5c-37adbcbf12a9","Type":"ContainerStarted","Data":"ba07637042f10f5c13d0d71430251677787b04042d7fd8c3e850d44abe405fc2"}
Sep 17 14:25:10.347801 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:10.347776    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:25:10.347854 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:10.347841    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:25:10.347887 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:10.347878    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:25:10.397654 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:10.396833    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46" podStartSLOduration=1.396809848 podStartE2EDuration="1.396809848s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:25:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:25:10.39635833 +0000 UTC m=+1211.397901806" watchObservedRunningTime="2026-09-17 14:25:10.396809848 +0000 UTC m=+1211.398353324"
Sep 17 14:25:11.360064 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:11.360025    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr" event={"ID":"932dad53-ff0a-412d-a874-55e0edc95702","Type":"ContainerStarted","Data":"4f11e69f4e1609715fb3b22835969aadcb007f572a47e85ad06145e6ca0cb248"}
Sep 17 14:25:11.700571 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:11.700544    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr"
Sep 17 14:25:11.700756 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:11.700731    2582 patch_prober.go:28] interesting pod/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Sep 17 14:25:11.700801 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:11.700780    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr" podUID="932dad53-ff0a-412d-a874-55e0edc95702" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Sep 17 14:25:12.700416 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:12.700375    2582 patch_prober.go:28] interesting pod/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Sep 17 14:25:12.700846 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:12.700469    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr" podUID="932dad53-ff0a-412d-a874-55e0edc95702" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Sep 17 14:25:13.700481 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:13.700446    2582 patch_prober.go:28] interesting pod/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Sep 17 14:25:13.700894 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:13.700504    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr" podUID="932dad53-ff0a-412d-a874-55e0edc95702" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Sep 17 14:25:14.118071 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:14.117978    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:25:14.700912 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:14.700877    2582 patch_prober.go:28] interesting pod/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Sep 17 14:25:14.701303 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:14.700967    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr" podUID="932dad53-ff0a-412d-a874-55e0edc95702" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Sep 17 14:25:14.774497 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:14.774455    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:25:15.562965 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:15.562928    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:25:15.700565 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:15.700532    2582 patch_prober.go:28] interesting pod/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Sep 17 14:25:15.700792 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:15.700641    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr" podUID="932dad53-ff0a-412d-a874-55e0edc95702" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Sep 17 14:25:16.238931 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:16.238893    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:25:16.752783 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:16.752690    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr"
Sep 17 14:25:16.753368 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:16.753336    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr"
Sep 17 14:25:16.754443 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:16.754406    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr"
Sep 17 14:25:17.226073 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:17.226025    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:25:17.907759 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:17.907719    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:25:18.293718 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:18.293681    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:25:18.572196 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:18.572113    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:25:19.055150 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:19.055114    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7865-44ef7b"]
Sep 17 14:25:19.057905 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:19.057887    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7865-44ef7b"
Sep 17 14:25:19.060137 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:19.060117    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-25mz9\""
Sep 17 14:25:19.065970 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:19.065946    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7865-44ef7b"]
Sep 17 14:25:19.091955 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:19.091913    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6fnh\" (UniqueName: \"kubernetes.io/projected/0f248e8d-496b-4b6b-ba60-6fd8229fb281-kube-api-access-c6fnh\") pod \"test-curl-7865-44ef7b\" (UID: \"0f248e8d-496b-4b6b-ba60-6fd8229fb281\") " pod="opendatahub/test-curl-7865-44ef7b"
Sep 17 14:25:19.193175 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:19.193135    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6fnh\" (UniqueName: \"kubernetes.io/projected/0f248e8d-496b-4b6b-ba60-6fd8229fb281-kube-api-access-c6fnh\") pod \"test-curl-7865-44ef7b\" (UID: \"0f248e8d-496b-4b6b-ba60-6fd8229fb281\") " pod="opendatahub/test-curl-7865-44ef7b"
Sep 17 14:25:19.202154 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:19.202108    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6fnh\" (UniqueName: \"kubernetes.io/projected/0f248e8d-496b-4b6b-ba60-6fd8229fb281-kube-api-access-c6fnh\") pod \"test-curl-7865-44ef7b\" (UID: \"0f248e8d-496b-4b6b-ba60-6fd8229fb281\") " pod="opendatahub/test-curl-7865-44ef7b"
Sep 17 14:25:19.369178 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:19.369087    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7865-44ef7b"
Sep 17 14:25:19.646730 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:19.646705    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7865-44ef7b"]
Sep 17 14:25:19.648461 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:25:19.648432    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0f248e8d_496b_4b6b_ba60_6fd8229fb281.slice/crio-ae6ea0d1ba0493c68afded91b2d12323072ed97345b15192faed6261222cc1f4 WatchSource:0}: Error finding container ae6ea0d1ba0493c68afded91b2d12323072ed97345b15192faed6261222cc1f4: Status 404 returned error can't find the container with id ae6ea0d1ba0493c68afded91b2d12323072ed97345b15192faed6261222cc1f4
Sep 17 14:25:19.901253 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:19.901167    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46"
Sep 17 14:25:19.901253 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:19.901220    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46"
Sep 17 14:25:19.914239 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:19.914212    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46"
Sep 17 14:25:20.419028 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:20.418991    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7865-44ef7b" event={"ID":"0f248e8d-496b-4b6b-ba60-6fd8229fb281","Type":"ContainerStarted","Data":"6cdc3b57cacc53c1fff96c7204fb59271eededc1d492f468924cf9d1c0b94554"}
Sep 17 14:25:20.419503 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:20.419038    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7865-44ef7b" event={"ID":"0f248e8d-496b-4b6b-ba60-6fd8229fb281","Type":"ContainerStarted","Data":"ae6ea0d1ba0493c68afded91b2d12323072ed97345b15192faed6261222cc1f4"}
Sep 17 14:25:20.432215 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:20.432190    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46"
Sep 17 14:25:20.437869 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:20.437822    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7865-44ef7b" podStartSLOduration=1.437808329 podStartE2EDuration="1.437808329s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:25:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:25:20.435027046 +0000 UTC m=+1221.436570521" watchObservedRunningTime="2026-09-17 14:25:20.437808329 +0000 UTC m=+1221.439351796"
Sep 17 14:25:21.034198 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:21.034154    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7865-44ef7b"]
Sep 17 14:25:21.038109 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:21.038074    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7865-44ef7b"]
Sep 17 14:25:21.161826 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:21.161742    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7865-540b3f"]
Sep 17 14:25:21.162284 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:21.162269    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f248e8d-496b-4b6b-ba60-6fd8229fb281" containerName="test-curl-7865-44ef7b"
Sep 17 14:25:21.162336 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:21.162285    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f248e8d-496b-4b6b-ba60-6fd8229fb281" containerName="test-curl-7865-44ef7b"
Sep 17 14:25:21.162391 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:21.162381    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f248e8d-496b-4b6b-ba60-6fd8229fb281" containerName="test-curl-7865-44ef7b"
Sep 17 14:25:21.164385 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:21.164368    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7865-540b3f"
Sep 17 14:25:21.170404 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:21.170380    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7865-540b3f"]
Sep 17 14:25:21.213663 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:21.213628    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9w7jx\" (UniqueName: \"kubernetes.io/projected/0843fca1-26da-47ea-97bd-94d7efb23471-kube-api-access-9w7jx\") pod \"test-curl-7865-540b3f\" (UID: \"0843fca1-26da-47ea-97bd-94d7efb23471\") " pod="opendatahub/test-curl-7865-540b3f"
Sep 17 14:25:21.315359 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:21.315309    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w7jx\" (UniqueName: \"kubernetes.io/projected/0843fca1-26da-47ea-97bd-94d7efb23471-kube-api-access-9w7jx\") pod \"test-curl-7865-540b3f\" (UID: \"0843fca1-26da-47ea-97bd-94d7efb23471\") " pod="opendatahub/test-curl-7865-540b3f"
Sep 17 14:25:21.324134 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:21.324105    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9w7jx\" (UniqueName: \"kubernetes.io/projected/0843fca1-26da-47ea-97bd-94d7efb23471-kube-api-access-9w7jx\") pod \"test-curl-7865-540b3f\" (UID: \"0843fca1-26da-47ea-97bd-94d7efb23471\") " pod="opendatahub/test-curl-7865-540b3f"
Sep 17 14:25:21.425357 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:25:21.425255    2582 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7865-44ef7b" container="test-curl-7865-44ef7b"
Sep 17 14:25:21.476444 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:21.476418    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7865-540b3f"
Sep 17 14:25:21.755007 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:21.754974    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7865-540b3f"]
Sep 17 14:25:21.756745 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:25:21.756709    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0843fca1_26da_47ea_97bd_94d7efb23471.slice/crio-068a1591a762a978f0cb5387673d693dbd0aaebf5b9ff98d2be3014ba569addc WatchSource:0}: Error finding container 068a1591a762a978f0cb5387673d693dbd0aaebf5b9ff98d2be3014ba569addc: Status 404 returned error can't find the container with id 068a1591a762a978f0cb5387673d693dbd0aaebf5b9ff98d2be3014ba569addc
Sep 17 14:25:22.434192 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:22.434147    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7865-540b3f" event={"ID":"0843fca1-26da-47ea-97bd-94d7efb23471","Type":"ContainerStarted","Data":"7310b91b4dbd38f22e8ad2ebdfd38312bd0d89d7a93ba6855dd7243765838030"}
Sep 17 14:25:22.434694 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:22.434200    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7865-540b3f" event={"ID":"0843fca1-26da-47ea-97bd-94d7efb23471","Type":"ContainerStarted","Data":"068a1591a762a978f0cb5387673d693dbd0aaebf5b9ff98d2be3014ba569addc"}
Sep 17 14:25:22.434694 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:22.434381    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7865-44ef7b" podUID="0f248e8d-496b-4b6b-ba60-6fd8229fb281" containerName="test-curl-7865-44ef7b" containerID="cri-o://6cdc3b57cacc53c1fff96c7204fb59271eededc1d492f468924cf9d1c0b94554" gracePeriod=2
Sep 17 14:25:22.451640 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:22.451578    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7865-540b3f" podStartSLOduration=1.451562001 podStartE2EDuration="1.451562001s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:25:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:25:22.450008937 +0000 UTC m=+1223.451552411" watchObservedRunningTime="2026-09-17 14:25:22.451562001 +0000 UTC m=+1223.453105475"
Sep 17 14:25:23.077805 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:23.077769    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7865-540b3f"]
Sep 17 14:25:23.080747 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:23.080713    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7865-540b3f"]
Sep 17 14:25:23.197103 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:23.197073    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7865-4e2744"]
Sep 17 14:25:23.197630 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:23.197596    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0843fca1-26da-47ea-97bd-94d7efb23471" containerName="test-curl-7865-540b3f"
Sep 17 14:25:23.197630 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:23.197627    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="0843fca1-26da-47ea-97bd-94d7efb23471" containerName="test-curl-7865-540b3f"
Sep 17 14:25:23.197758 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:23.197748    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="0843fca1-26da-47ea-97bd-94d7efb23471" containerName="test-curl-7865-540b3f"
Sep 17 14:25:23.199807 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:23.199793    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7865-4e2744"
Sep 17 14:25:23.207238 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:23.207213    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7865-4e2744"]
Sep 17 14:25:23.237212 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:23.237167    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcr27\" (UniqueName: \"kubernetes.io/projected/2cdce343-145c-418c-80bf-70cd13ae8e75-kube-api-access-lcr27\") pod \"test-curl-7865-4e2744\" (UID: \"2cdce343-145c-418c-80bf-70cd13ae8e75\") " pod="opendatahub/test-curl-7865-4e2744"
Sep 17 14:25:23.338250 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:23.338210    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcr27\" (UniqueName: \"kubernetes.io/projected/2cdce343-145c-418c-80bf-70cd13ae8e75-kube-api-access-lcr27\") pod \"test-curl-7865-4e2744\" (UID: \"2cdce343-145c-418c-80bf-70cd13ae8e75\") " pod="opendatahub/test-curl-7865-4e2744"
Sep 17 14:25:23.347015 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:23.346976    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcr27\" (UniqueName: \"kubernetes.io/projected/2cdce343-145c-418c-80bf-70cd13ae8e75-kube-api-access-lcr27\") pod \"test-curl-7865-4e2744\" (UID: \"2cdce343-145c-418c-80bf-70cd13ae8e75\") " pod="opendatahub/test-curl-7865-4e2744"
Sep 17 14:25:23.440931 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:25:23.440897    2582 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7865-540b3f" container="test-curl-7865-540b3f"
Sep 17 14:25:23.512082 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:23.511991    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7865-4e2744"
Sep 17 14:25:23.788131 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:23.788104    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7865-4e2744"]
Sep 17 14:25:23.789893 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:25:23.789866    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2cdce343_145c_418c_80bf_70cd13ae8e75.slice/crio-1f52491182de22f2e0aac4bedd6ff35478ce9faabe8ccd166a7bf07ca1e8f531 WatchSource:0}: Error finding container 1f52491182de22f2e0aac4bedd6ff35478ce9faabe8ccd166a7bf07ca1e8f531: Status 404 returned error can't find the container with id 1f52491182de22f2e0aac4bedd6ff35478ce9faabe8ccd166a7bf07ca1e8f531
Sep 17 14:25:24.448651 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:24.448622    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7865-4e2744" event={"ID":"2cdce343-145c-418c-80bf-70cd13ae8e75","Type":"ContainerStarted","Data":"fce9f3dc2e550f0af29f5147efcfb835be355d396fb82ee0202bd77cdb346a33"}
Sep 17 14:25:24.449028 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:24.448661    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7865-4e2744" event={"ID":"2cdce343-145c-418c-80bf-70cd13ae8e75","Type":"ContainerStarted","Data":"1f52491182de22f2e0aac4bedd6ff35478ce9faabe8ccd166a7bf07ca1e8f531"}
Sep 17 14:25:24.449028 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:24.448756    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7865-540b3f" podUID="0843fca1-26da-47ea-97bd-94d7efb23471" containerName="test-curl-7865-540b3f" containerID="cri-o://7310b91b4dbd38f22e8ad2ebdfd38312bd0d89d7a93ba6855dd7243765838030" gracePeriod=2
Sep 17 14:25:24.465102 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:24.465050    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7865-4e2744" podStartSLOduration=1.4650320350000001 podStartE2EDuration="1.465032035s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:25:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:25:24.463065612 +0000 UTC m=+1225.464609085" watchObservedRunningTime="2026-09-17 14:25:24.465032035 +0000 UTC m=+1225.466575509"
Sep 17 14:25:24.618023 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:24.617997    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7865-44ef7b"
Sep 17 14:25:24.754781 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:24.754747    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c6fnh\" (UniqueName: \"kubernetes.io/projected/0f248e8d-496b-4b6b-ba60-6fd8229fb281-kube-api-access-c6fnh\") pod \"0f248e8d-496b-4b6b-ba60-6fd8229fb281\" (UID: \"0f248e8d-496b-4b6b-ba60-6fd8229fb281\") "
Sep 17 14:25:24.756786 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:24.756762    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f248e8d-496b-4b6b-ba60-6fd8229fb281-kube-api-access-c6fnh" (OuterVolumeSpecName: "kube-api-access-c6fnh") pod "0f248e8d-496b-4b6b-ba60-6fd8229fb281" (UID: "0f248e8d-496b-4b6b-ba60-6fd8229fb281"). InnerVolumeSpecName "kube-api-access-c6fnh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:25:24.856179 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:24.856151    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c6fnh\" (UniqueName: \"kubernetes.io/projected/0f248e8d-496b-4b6b-ba60-6fd8229fb281-kube-api-access-c6fnh\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:25:25.065758 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:25.065674    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7865-4e2744"]
Sep 17 14:25:25.069542 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:25.069508    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7865-4e2744"]
Sep 17 14:25:25.185644 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:25.181250    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7865-e090b7"]
Sep 17 14:25:25.185644 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:25.182828    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cdce343-145c-418c-80bf-70cd13ae8e75" containerName="test-curl-7865-4e2744"
Sep 17 14:25:25.185644 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:25.182868    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cdce343-145c-418c-80bf-70cd13ae8e75" containerName="test-curl-7865-4e2744"
Sep 17 14:25:25.185644 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:25.183129    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2cdce343-145c-418c-80bf-70cd13ae8e75" containerName="test-curl-7865-4e2744"
Sep 17 14:25:25.187518 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:25.186942    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7865-e090b7"
Sep 17 14:25:25.193694 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:25.193666    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7865-e090b7"]
Sep 17 14:25:25.260307 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:25.260273    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59sjh\" (UniqueName: \"kubernetes.io/projected/7ce98672-2e99-42a4-b090-c65ac8470256-kube-api-access-59sjh\") pod \"test-curl-7865-e090b7\" (UID: \"7ce98672-2e99-42a4-b090-c65ac8470256\") " pod="opendatahub/test-curl-7865-e090b7"
Sep 17 14:25:25.361912 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:25.361819    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59sjh\" (UniqueName: \"kubernetes.io/projected/7ce98672-2e99-42a4-b090-c65ac8470256-kube-api-access-59sjh\") pod \"test-curl-7865-e090b7\" (UID: \"7ce98672-2e99-42a4-b090-c65ac8470256\") " pod="opendatahub/test-curl-7865-e090b7"
Sep 17 14:25:25.371174 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:25.371142    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59sjh\" (UniqueName: \"kubernetes.io/projected/7ce98672-2e99-42a4-b090-c65ac8470256-kube-api-access-59sjh\") pod \"test-curl-7865-e090b7\" (UID: \"7ce98672-2e99-42a4-b090-c65ac8470256\") " pod="opendatahub/test-curl-7865-e090b7"
Sep 17 14:25:25.456158 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:25.456118    2582 generic.go:356] "Generic (PLEG): container finished" podID="0f248e8d-496b-4b6b-ba60-6fd8229fb281" containerID="6cdc3b57cacc53c1fff96c7204fb59271eededc1d492f468924cf9d1c0b94554" exitCode=137
Sep 17 14:25:25.456562 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:25.456173    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7865-44ef7b"
Sep 17 14:25:25.456562 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:25.456175    2582 scope.go:117] "RemoveContainer" containerID="6cdc3b57cacc53c1fff96c7204fb59271eededc1d492f468924cf9d1c0b94554"
Sep 17 14:25:25.456562 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:25:25.456528    2582 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7865-4e2744" container="test-curl-7865-4e2744"
Sep 17 14:25:25.466259 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:25.466241    2582 scope.go:117] "RemoveContainer" containerID="6cdc3b57cacc53c1fff96c7204fb59271eededc1d492f468924cf9d1c0b94554"
Sep 17 14:25:25.466527 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:25:25.466505    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6cdc3b57cacc53c1fff96c7204fb59271eededc1d492f468924cf9d1c0b94554\": container with ID starting with 6cdc3b57cacc53c1fff96c7204fb59271eededc1d492f468924cf9d1c0b94554 not found: ID does not exist" containerID="6cdc3b57cacc53c1fff96c7204fb59271eededc1d492f468924cf9d1c0b94554"
Sep 17 14:25:25.466579 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:25.466538    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cdc3b57cacc53c1fff96c7204fb59271eededc1d492f468924cf9d1c0b94554"} err="failed to get container status \"6cdc3b57cacc53c1fff96c7204fb59271eededc1d492f468924cf9d1c0b94554\": rpc error: code = NotFound desc = could not find container \"6cdc3b57cacc53c1fff96c7204fb59271eededc1d492f468924cf9d1c0b94554\": container with ID starting with 6cdc3b57cacc53c1fff96c7204fb59271eededc1d492f468924cf9d1c0b94554 not found: ID does not exist"
Sep 17 14:25:25.467354 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:25:25.467331    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7865-44ef7b\" is forbidden: User \"system:node:ip-10-0-137-53.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-53.ec2.internal' and this object" podUID="0f248e8d-496b-4b6b-ba60-6fd8229fb281" pod="opendatahub/test-curl-7865-44ef7b"
Sep 17 14:25:25.494758 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:25.494732    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f248e8d-496b-4b6b-ba60-6fd8229fb281" path="/var/lib/kubelet/pods/0f248e8d-496b-4b6b-ba60-6fd8229fb281/volumes"
Sep 17 14:25:25.500617 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:25.500591    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7865-e090b7"
Sep 17 14:25:25.773567 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:25.773412    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7865-e090b7"]
Sep 17 14:25:25.778094 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:25:25.778053    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7ce98672_2e99_42a4_b090_c65ac8470256.slice/crio-116670b0e7fb6006821cb6cec5cd7ce1218410252fa1f65114d59872000cd5d7 WatchSource:0}: Error finding container 116670b0e7fb6006821cb6cec5cd7ce1218410252fa1f65114d59872000cd5d7: Status 404 returned error can't find the container with id 116670b0e7fb6006821cb6cec5cd7ce1218410252fa1f65114d59872000cd5d7
Sep 17 14:25:26.466370 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:26.466338    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7865-e090b7" event={"ID":"7ce98672-2e99-42a4-b090-c65ac8470256","Type":"ContainerStarted","Data":"ee9ccc53460983c7cfa7cdf24a6fc144bd448ad38227273d538295e6d790149d"}
Sep 17 14:25:26.466752 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:26.466384    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7865-e090b7" event={"ID":"7ce98672-2e99-42a4-b090-c65ac8470256","Type":"ContainerStarted","Data":"116670b0e7fb6006821cb6cec5cd7ce1218410252fa1f65114d59872000cd5d7"}
Sep 17 14:25:26.466752 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:26.466493    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7865-4e2744" podUID="2cdce343-145c-418c-80bf-70cd13ae8e75" containerName="test-curl-7865-4e2744" containerID="cri-o://fce9f3dc2e550f0af29f5147efcfb835be355d396fb82ee0202bd77cdb346a33" gracePeriod=2
Sep 17 14:25:26.484788 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:26.484732    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7865-e090b7" podStartSLOduration=1.484708575 podStartE2EDuration="1.484708575s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:25:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:25:26.481173031 +0000 UTC m=+1227.482716505" watchObservedRunningTime="2026-09-17 14:25:26.484708575 +0000 UTC m=+1227.486252049"
Sep 17 14:25:26.617319 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:26.617287    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7865-540b3f"
Sep 17 14:25:26.675860 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:26.675829    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9w7jx\" (UniqueName: \"kubernetes.io/projected/0843fca1-26da-47ea-97bd-94d7efb23471-kube-api-access-9w7jx\") pod \"0843fca1-26da-47ea-97bd-94d7efb23471\" (UID: \"0843fca1-26da-47ea-97bd-94d7efb23471\") "
Sep 17 14:25:26.678171 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:26.678128    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0843fca1-26da-47ea-97bd-94d7efb23471-kube-api-access-9w7jx" (OuterVolumeSpecName: "kube-api-access-9w7jx") pod "0843fca1-26da-47ea-97bd-94d7efb23471" (UID: "0843fca1-26da-47ea-97bd-94d7efb23471"). InnerVolumeSpecName "kube-api-access-9w7jx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:25:26.777454 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:26.777416    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9w7jx\" (UniqueName: \"kubernetes.io/projected/0843fca1-26da-47ea-97bd-94d7efb23471-kube-api-access-9w7jx\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:25:27.062270 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:27.062184    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7865-e090b7"]
Sep 17 14:25:27.066107 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:27.066080    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7865-e090b7"]
Sep 17 14:25:27.474115 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:27.474078    2582 generic.go:356] "Generic (PLEG): container finished" podID="0843fca1-26da-47ea-97bd-94d7efb23471" containerID="7310b91b4dbd38f22e8ad2ebdfd38312bd0d89d7a93ba6855dd7243765838030" exitCode=137
Sep 17 14:25:27.474568 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:27.474119    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7865-540b3f"
Sep 17 14:25:27.474568 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:27.474179    2582 scope.go:117] "RemoveContainer" containerID="7310b91b4dbd38f22e8ad2ebdfd38312bd0d89d7a93ba6855dd7243765838030"
Sep 17 14:25:27.474568 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:25:27.474449    2582 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7865-e090b7" container="test-curl-7865-e090b7"
Sep 17 14:25:27.483521 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:27.483504    2582 scope.go:117] "RemoveContainer" containerID="7310b91b4dbd38f22e8ad2ebdfd38312bd0d89d7a93ba6855dd7243765838030"
Sep 17 14:25:27.483807 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:25:27.483792    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7310b91b4dbd38f22e8ad2ebdfd38312bd0d89d7a93ba6855dd7243765838030\": container with ID starting with 7310b91b4dbd38f22e8ad2ebdfd38312bd0d89d7a93ba6855dd7243765838030 not found: ID does not exist" containerID="7310b91b4dbd38f22e8ad2ebdfd38312bd0d89d7a93ba6855dd7243765838030"
Sep 17 14:25:27.483887 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:27.483825    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7310b91b4dbd38f22e8ad2ebdfd38312bd0d89d7a93ba6855dd7243765838030"} err="failed to get container status \"7310b91b4dbd38f22e8ad2ebdfd38312bd0d89d7a93ba6855dd7243765838030\": rpc error: code = NotFound desc = could not find container \"7310b91b4dbd38f22e8ad2ebdfd38312bd0d89d7a93ba6855dd7243765838030\": container with ID starting with 7310b91b4dbd38f22e8ad2ebdfd38312bd0d89d7a93ba6855dd7243765838030 not found: ID does not exist"
Sep 17 14:25:27.484538 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:25:27.484515    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7865-540b3f\" is forbidden: User \"system:node:ip-10-0-137-53.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-53.ec2.internal' and this object" podUID="0843fca1-26da-47ea-97bd-94d7efb23471" pod="opendatahub/test-curl-7865-540b3f"
Sep 17 14:25:27.493782 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:27.493762    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0843fca1-26da-47ea-97bd-94d7efb23471" path="/var/lib/kubelet/pods/0843fca1-26da-47ea-97bd-94d7efb23471/volumes"
Sep 17 14:25:27.761017 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:27.760926    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:25:27.937185 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:27.937151    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn"]
Sep 17 14:25:27.937832 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:27.937810    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ce98672-2e99-42a4-b090-c65ac8470256" containerName="test-curl-7865-e090b7"
Sep 17 14:25:27.937832 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:27.937830    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ce98672-2e99-42a4-b090-c65ac8470256" containerName="test-curl-7865-e090b7"
Sep 17 14:25:27.938022 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:27.937969    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ce98672-2e99-42a4-b090-c65ac8470256" containerName="test-curl-7865-e090b7"
Sep 17 14:25:27.941319 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:27.941300    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn"
Sep 17 14:25:27.943826 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:27.943795    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-1a095586-openshift-default-dockercfg-t5n26\""
Sep 17 14:25:27.954167 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:27.954134    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn"]
Sep 17 14:25:27.990598 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:27.990561    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-credential-socket\") pod \"e2e-mt-1a095586-openshift-default-df7756fbd-d2frn\" (UID: \"3ab0e2f8-287d-4ba7-b06c-e3bef8457239\") " pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn"
Sep 17 14:25:27.990774 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:27.990649    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-istio-token\") pod \"e2e-mt-1a095586-openshift-default-df7756fbd-d2frn\" (UID: \"3ab0e2f8-287d-4ba7-b06c-e3bef8457239\") " pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn"
Sep 17 14:25:27.990774 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:27.990674    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-istio-envoy\") pod \"e2e-mt-1a095586-openshift-default-df7756fbd-d2frn\" (UID: \"3ab0e2f8-287d-4ba7-b06c-e3bef8457239\") " pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn"
Sep 17 14:25:27.990774 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:27.990751    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-workload-certs\") pod \"e2e-mt-1a095586-openshift-default-df7756fbd-d2frn\" (UID: \"3ab0e2f8-287d-4ba7-b06c-e3bef8457239\") " pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn"
Sep 17 14:25:27.990880 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:27.990788    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-istio-data\") pod \"e2e-mt-1a095586-openshift-default-df7756fbd-d2frn\" (UID: \"3ab0e2f8-287d-4ba7-b06c-e3bef8457239\") " pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn"
Sep 17 14:25:27.990880 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:27.990814    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8q5w\" (UniqueName: \"kubernetes.io/projected/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-kube-api-access-c8q5w\") pod \"e2e-mt-1a095586-openshift-default-df7756fbd-d2frn\" (UID: \"3ab0e2f8-287d-4ba7-b06c-e3bef8457239\") " pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn"
Sep 17 14:25:27.990880 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:27.990843    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-istio-podinfo\") pod \"e2e-mt-1a095586-openshift-default-df7756fbd-d2frn\" (UID: \"3ab0e2f8-287d-4ba7-b06c-e3bef8457239\") " pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn"
Sep 17 14:25:27.990880 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:27.990877    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-workload-socket\") pod \"e2e-mt-1a095586-openshift-default-df7756fbd-d2frn\" (UID: \"3ab0e2f8-287d-4ba7-b06c-e3bef8457239\") " pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn"
Sep 17 14:25:27.991006 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:27.990930    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-istiod-ca-cert\") pod \"e2e-mt-1a095586-openshift-default-df7756fbd-d2frn\" (UID: \"3ab0e2f8-287d-4ba7-b06c-e3bef8457239\") " pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn"
Sep 17 14:25:28.092265 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:28.092171    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-istio-data\") pod \"e2e-mt-1a095586-openshift-default-df7756fbd-d2frn\" (UID: \"3ab0e2f8-287d-4ba7-b06c-e3bef8457239\") " pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn"
Sep 17 14:25:28.092265 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:28.092215    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8q5w\" (UniqueName: \"kubernetes.io/projected/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-kube-api-access-c8q5w\") pod \"e2e-mt-1a095586-openshift-default-df7756fbd-d2frn\" (UID: \"3ab0e2f8-287d-4ba7-b06c-e3bef8457239\") " pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn"
Sep 17 14:25:28.092265 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:28.092236    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-istio-podinfo\") pod \"e2e-mt-1a095586-openshift-default-df7756fbd-d2frn\" (UID: \"3ab0e2f8-287d-4ba7-b06c-e3bef8457239\") " pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn"
Sep 17 14:25:28.092265 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:28.092257    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-workload-socket\") pod \"e2e-mt-1a095586-openshift-default-df7756fbd-d2frn\" (UID: \"3ab0e2f8-287d-4ba7-b06c-e3bef8457239\") " pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn"
Sep 17 14:25:28.092660 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:28.092321    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-istiod-ca-cert\") pod \"e2e-mt-1a095586-openshift-default-df7756fbd-d2frn\" (UID: \"3ab0e2f8-287d-4ba7-b06c-e3bef8457239\") " pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn"
Sep 17 14:25:28.092660 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:28.092354    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-credential-socket\") pod \"e2e-mt-1a095586-openshift-default-df7756fbd-d2frn\" (UID: \"3ab0e2f8-287d-4ba7-b06c-e3bef8457239\") " pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn"
Sep 17 14:25:28.092660 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:28.092419    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-istio-token\") pod \"e2e-mt-1a095586-openshift-default-df7756fbd-d2frn\" (UID: \"3ab0e2f8-287d-4ba7-b06c-e3bef8457239\") " pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn"
Sep 17 14:25:28.092660 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:28.092449    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-istio-envoy\") pod \"e2e-mt-1a095586-openshift-default-df7756fbd-d2frn\" (UID: \"3ab0e2f8-287d-4ba7-b06c-e3bef8457239\") " pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn"
Sep 17 14:25:28.092660 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:28.092521    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-workload-certs\") pod \"e2e-mt-1a095586-openshift-default-df7756fbd-d2frn\" (UID: \"3ab0e2f8-287d-4ba7-b06c-e3bef8457239\") " pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn"
Sep 17 14:25:28.092873 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:28.092707    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-istio-data\") pod \"e2e-mt-1a095586-openshift-default-df7756fbd-d2frn\" (UID: \"3ab0e2f8-287d-4ba7-b06c-e3bef8457239\") " pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn"
Sep 17 14:25:28.092873 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:28.092837    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-credential-socket\") pod \"e2e-mt-1a095586-openshift-default-df7756fbd-d2frn\" (UID: \"3ab0e2f8-287d-4ba7-b06c-e3bef8457239\") " pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn"
Sep 17 14:25:28.092961 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:28.092905    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-workload-certs\") pod \"e2e-mt-1a095586-openshift-default-df7756fbd-d2frn\" (UID: \"3ab0e2f8-287d-4ba7-b06c-e3bef8457239\") " pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn"
Sep 17 14:25:28.093045 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:28.093029    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-workload-socket\") pod \"e2e-mt-1a095586-openshift-default-df7756fbd-d2frn\" (UID: \"3ab0e2f8-287d-4ba7-b06c-e3bef8457239\") " pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn"
Sep 17 14:25:28.093125 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:28.093102    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-istiod-ca-cert\") pod \"e2e-mt-1a095586-openshift-default-df7756fbd-d2frn\" (UID: \"3ab0e2f8-287d-4ba7-b06c-e3bef8457239\") " pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn"
Sep 17 14:25:28.094718 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:28.094698    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-istio-envoy\") pod \"e2e-mt-1a095586-openshift-default-df7756fbd-d2frn\" (UID: \"3ab0e2f8-287d-4ba7-b06c-e3bef8457239\") " pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn"
Sep 17 14:25:28.095066 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:28.095046    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-istio-podinfo\") pod \"e2e-mt-1a095586-openshift-default-df7756fbd-d2frn\" (UID: \"3ab0e2f8-287d-4ba7-b06c-e3bef8457239\") " pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn"
Sep 17 14:25:28.100129 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:28.100098    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-istio-token\") pod \"e2e-mt-1a095586-openshift-default-df7756fbd-d2frn\" (UID: \"3ab0e2f8-287d-4ba7-b06c-e3bef8457239\") " pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn"
Sep 17 14:25:28.100257 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:28.100243    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8q5w\" (UniqueName: \"kubernetes.io/projected/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-kube-api-access-c8q5w\") pod \"e2e-mt-1a095586-openshift-default-df7756fbd-d2frn\" (UID: \"3ab0e2f8-287d-4ba7-b06c-e3bef8457239\") " pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn"
Sep 17 14:25:28.254724 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:28.254684    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn"
Sep 17 14:25:28.482242 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:28.482210    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7865-e090b7" podUID="7ce98672-2e99-42a4-b090-c65ac8470256" containerName="test-curl-7865-e090b7" containerID="cri-o://ee9ccc53460983c7cfa7cdf24a6fc144bd448ad38227273d538295e6d790149d" gracePeriod=2
Sep 17 14:25:28.532945 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:28.532912    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn"]
Sep 17 14:25:28.534834 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:25:28.534804    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ab0e2f8_287d_4ba7_b06c_e3bef8457239.slice/crio-cecc4a52693e4cfbf0fcad202d6aa367456a3c2de5debcd4242dd9cc421017bf WatchSource:0}: Error finding container cecc4a52693e4cfbf0fcad202d6aa367456a3c2de5debcd4242dd9cc421017bf: Status 404 returned error can't find the container with id cecc4a52693e4cfbf0fcad202d6aa367456a3c2de5debcd4242dd9cc421017bf
Sep 17 14:25:28.537564 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:28.537523    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:25:28.537682 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:28.537634    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:25:28.537738 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:28.537681    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:25:28.618340 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:28.618305    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7865-4e2744"
Sep 17 14:25:28.700027 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:28.699990    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lcr27\" (UniqueName: \"kubernetes.io/projected/2cdce343-145c-418c-80bf-70cd13ae8e75-kube-api-access-lcr27\") pod \"2cdce343-145c-418c-80bf-70cd13ae8e75\" (UID: \"2cdce343-145c-418c-80bf-70cd13ae8e75\") "
Sep 17 14:25:28.702168 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:28.702134    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cdce343-145c-418c-80bf-70cd13ae8e75-kube-api-access-lcr27" (OuterVolumeSpecName: "kube-api-access-lcr27") pod "2cdce343-145c-418c-80bf-70cd13ae8e75" (UID: "2cdce343-145c-418c-80bf-70cd13ae8e75"). InnerVolumeSpecName "kube-api-access-lcr27". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:25:28.801357 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:28.801267    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lcr27\" (UniqueName: \"kubernetes.io/projected/2cdce343-145c-418c-80bf-70cd13ae8e75-kube-api-access-lcr27\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:25:29.333084 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:29.333033    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:25:29.489522 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:29.489490    2582 generic.go:356] "Generic (PLEG): container finished" podID="2cdce343-145c-418c-80bf-70cd13ae8e75" containerID="fce9f3dc2e550f0af29f5147efcfb835be355d396fb82ee0202bd77cdb346a33" exitCode=137
Sep 17 14:25:29.491096 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:29.491064    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7865-4e2744"
Sep 17 14:25:29.503984 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:29.503954    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2cdce343-145c-418c-80bf-70cd13ae8e75" path="/var/lib/kubelet/pods/2cdce343-145c-418c-80bf-70cd13ae8e75/volumes"
Sep 17 14:25:29.504428 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:29.504405    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn" event={"ID":"3ab0e2f8-287d-4ba7-b06c-e3bef8457239","Type":"ContainerStarted","Data":"015b660060515c2ef0e8448611ac637a732cd9c330dd99d69e03445a60aab678"}
Sep 17 14:25:29.504516 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:29.504449    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn" event={"ID":"3ab0e2f8-287d-4ba7-b06c-e3bef8457239","Type":"ContainerStarted","Data":"cecc4a52693e4cfbf0fcad202d6aa367456a3c2de5debcd4242dd9cc421017bf"}
Sep 17 14:25:29.504516 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:29.504474    2582 scope.go:117] "RemoveContainer" containerID="fce9f3dc2e550f0af29f5147efcfb835be355d396fb82ee0202bd77cdb346a33"
Sep 17 14:25:29.514484 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:29.514461    2582 scope.go:117] "RemoveContainer" containerID="fce9f3dc2e550f0af29f5147efcfb835be355d396fb82ee0202bd77cdb346a33"
Sep 17 14:25:29.514833 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:25:29.514810    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fce9f3dc2e550f0af29f5147efcfb835be355d396fb82ee0202bd77cdb346a33\": container with ID starting with fce9f3dc2e550f0af29f5147efcfb835be355d396fb82ee0202bd77cdb346a33 not found: ID does not exist" containerID="fce9f3dc2e550f0af29f5147efcfb835be355d396fb82ee0202bd77cdb346a33"
Sep 17 14:25:29.514905 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:29.514845    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fce9f3dc2e550f0af29f5147efcfb835be355d396fb82ee0202bd77cdb346a33"} err="failed to get container status \"fce9f3dc2e550f0af29f5147efcfb835be355d396fb82ee0202bd77cdb346a33\": rpc error: code = NotFound desc = could not find container \"fce9f3dc2e550f0af29f5147efcfb835be355d396fb82ee0202bd77cdb346a33\": container with ID starting with fce9f3dc2e550f0af29f5147efcfb835be355d396fb82ee0202bd77cdb346a33 not found: ID does not exist"
Sep 17 14:25:29.521711 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:29.521664    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn" podStartSLOduration=2.521650584 podStartE2EDuration="2.521650584s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:25:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:25:29.515690033 +0000 UTC m=+1230.517233510" watchObservedRunningTime="2026-09-17 14:25:29.521650584 +0000 UTC m=+1230.523194057"
Sep 17 14:25:30.255819 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:30.255779    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn"
Sep 17 14:25:30.261208 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:30.261180    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn"
Sep 17 14:25:30.501105 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:30.501079    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn"
Sep 17 14:25:30.501981 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:30.501961    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn"
Sep 17 14:25:30.653053 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:30.653021    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7865-e090b7"
Sep 17 14:25:30.723256 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:30.723225    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-59sjh\" (UniqueName: \"kubernetes.io/projected/7ce98672-2e99-42a4-b090-c65ac8470256-kube-api-access-59sjh\") pod \"7ce98672-2e99-42a4-b090-c65ac8470256\" (UID: \"7ce98672-2e99-42a4-b090-c65ac8470256\") "
Sep 17 14:25:30.725345 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:30.725312    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ce98672-2e99-42a4-b090-c65ac8470256-kube-api-access-59sjh" (OuterVolumeSpecName: "kube-api-access-59sjh") pod "7ce98672-2e99-42a4-b090-c65ac8470256" (UID: "7ce98672-2e99-42a4-b090-c65ac8470256"). InnerVolumeSpecName "kube-api-access-59sjh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:25:30.729657 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:30.729583    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:25:30.825188 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:30.825099    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-59sjh\" (UniqueName: \"kubernetes.io/projected/7ce98672-2e99-42a4-b090-c65ac8470256-kube-api-access-59sjh\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:25:31.448425 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:31.448388    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:25:31.495030 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:31.495001    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ce98672-2e99-42a4-b090-c65ac8470256" path="/var/lib/kubelet/pods/7ce98672-2e99-42a4-b090-c65ac8470256/volumes"
Sep 17 14:25:31.508582 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:31.508555    2582 generic.go:356] "Generic (PLEG): container finished" podID="7ce98672-2e99-42a4-b090-c65ac8470256" containerID="ee9ccc53460983c7cfa7cdf24a6fc144bd448ad38227273d538295e6d790149d" exitCode=137
Sep 17 14:25:31.508971 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:31.508593    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7865-e090b7"
Sep 17 14:25:31.508971 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:31.508678    2582 scope.go:117] "RemoveContainer" containerID="ee9ccc53460983c7cfa7cdf24a6fc144bd448ad38227273d538295e6d790149d"
Sep 17 14:25:31.518415 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:31.518398    2582 scope.go:117] "RemoveContainer" containerID="ee9ccc53460983c7cfa7cdf24a6fc144bd448ad38227273d538295e6d790149d"
Sep 17 14:25:31.518690 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:25:31.518668    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee9ccc53460983c7cfa7cdf24a6fc144bd448ad38227273d538295e6d790149d\": container with ID starting with ee9ccc53460983c7cfa7cdf24a6fc144bd448ad38227273d538295e6d790149d not found: ID does not exist" containerID="ee9ccc53460983c7cfa7cdf24a6fc144bd448ad38227273d538295e6d790149d"
Sep 17 14:25:31.518769 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:31.518704    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee9ccc53460983c7cfa7cdf24a6fc144bd448ad38227273d538295e6d790149d"} err="failed to get container status \"ee9ccc53460983c7cfa7cdf24a6fc144bd448ad38227273d538295e6d790149d\": rpc error: code = NotFound desc = could not find container \"ee9ccc53460983c7cfa7cdf24a6fc144bd448ad38227273d538295e6d790149d\": container with ID starting with ee9ccc53460983c7cfa7cdf24a6fc144bd448ad38227273d538295e6d790149d not found: ID does not exist"
Sep 17 14:25:31.550943 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:31.550904    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:25:32.338717 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:32.338686    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:25:33.161355 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:33.161319    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:25:33.878186 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:33.878131    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:25:34.704969 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:34.704933    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:25:35.448594 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:35.448557    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:25:36.174464 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:36.174433    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:25:36.251539 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:36.251502    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:25:36.438796 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:36.438696    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:25:36.618006 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:36.617970    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:25:36.719564 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:36.719529    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:25:50.305147 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:50.305110    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-0c1110/noten-model-0c1110-kserve-5797796699-d6ssx"]
Sep 17 14:25:50.308563 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:50.308538    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-0c1110/noten-model-0c1110-kserve-5797796699-d6ssx"
Sep 17 14:25:50.312182 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:50.312155    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-0c1110\"/\"default-dockercfg-vhl2r\""
Sep 17 14:25:50.312182 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:50.312175    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-0c1110\"/\"kube-root-ca.crt\""
Sep 17 14:25:50.312382 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:50.312191    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-0c1110\"/\"openshift-service-ca.crt\""
Sep 17 14:25:50.312382 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:50.312155    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-0c1110\"/\"noten-model-0c1110-kserve-self-signed-certs\""
Sep 17 14:25:50.320070 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:50.320046    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-0c1110/noten-model-0c1110-kserve-5797796699-d6ssx"]
Sep 17 14:25:50.335885 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:50.335843    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtpqn\" (UniqueName: \"kubernetes.io/projected/341bef6c-9dd7-4f4c-b1cc-dc69e421ab98-kube-api-access-rtpqn\") pod \"noten-model-0c1110-kserve-5797796699-d6ssx\" (UID: \"341bef6c-9dd7-4f4c-b1cc-dc69e421ab98\") " pod="ai-tenant-e2e-noten-0c1110/noten-model-0c1110-kserve-5797796699-d6ssx"
Sep 17 14:25:50.336059 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:50.335927    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/341bef6c-9dd7-4f4c-b1cc-dc69e421ab98-tmp-dir\") pod \"noten-model-0c1110-kserve-5797796699-d6ssx\" (UID: \"341bef6c-9dd7-4f4c-b1cc-dc69e421ab98\") " pod="ai-tenant-e2e-noten-0c1110/noten-model-0c1110-kserve-5797796699-d6ssx"
Sep 17 14:25:50.336059 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:50.335980    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/341bef6c-9dd7-4f4c-b1cc-dc69e421ab98-home\") pod \"noten-model-0c1110-kserve-5797796699-d6ssx\" (UID: \"341bef6c-9dd7-4f4c-b1cc-dc69e421ab98\") " pod="ai-tenant-e2e-noten-0c1110/noten-model-0c1110-kserve-5797796699-d6ssx"
Sep 17 14:25:50.336059 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:50.336028    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/341bef6c-9dd7-4f4c-b1cc-dc69e421ab98-tls-certs\") pod \"noten-model-0c1110-kserve-5797796699-d6ssx\" (UID: \"341bef6c-9dd7-4f4c-b1cc-dc69e421ab98\") " pod="ai-tenant-e2e-noten-0c1110/noten-model-0c1110-kserve-5797796699-d6ssx"
Sep 17 14:25:50.336231 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:50.336088    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/341bef6c-9dd7-4f4c-b1cc-dc69e421ab98-model-cache\") pod \"noten-model-0c1110-kserve-5797796699-d6ssx\" (UID: \"341bef6c-9dd7-4f4c-b1cc-dc69e421ab98\") " pod="ai-tenant-e2e-noten-0c1110/noten-model-0c1110-kserve-5797796699-d6ssx"
Sep 17 14:25:50.336231 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:50.336146    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/341bef6c-9dd7-4f4c-b1cc-dc69e421ab98-dshm\") pod \"noten-model-0c1110-kserve-5797796699-d6ssx\" (UID: \"341bef6c-9dd7-4f4c-b1cc-dc69e421ab98\") " pod="ai-tenant-e2e-noten-0c1110/noten-model-0c1110-kserve-5797796699-d6ssx"
Sep 17 14:25:50.336982 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:50.336961    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:25:50.437689 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:50.437650    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/341bef6c-9dd7-4f4c-b1cc-dc69e421ab98-home\") pod \"noten-model-0c1110-kserve-5797796699-d6ssx\" (UID: \"341bef6c-9dd7-4f4c-b1cc-dc69e421ab98\") " pod="ai-tenant-e2e-noten-0c1110/noten-model-0c1110-kserve-5797796699-d6ssx"
Sep 17 14:25:50.437890 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:50.437714    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/341bef6c-9dd7-4f4c-b1cc-dc69e421ab98-tls-certs\") pod \"noten-model-0c1110-kserve-5797796699-d6ssx\" (UID: \"341bef6c-9dd7-4f4c-b1cc-dc69e421ab98\") " pod="ai-tenant-e2e-noten-0c1110/noten-model-0c1110-kserve-5797796699-d6ssx"
Sep 17 14:25:50.437890 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:50.437763    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/341bef6c-9dd7-4f4c-b1cc-dc69e421ab98-model-cache\") pod \"noten-model-0c1110-kserve-5797796699-d6ssx\" (UID: \"341bef6c-9dd7-4f4c-b1cc-dc69e421ab98\") " pod="ai-tenant-e2e-noten-0c1110/noten-model-0c1110-kserve-5797796699-d6ssx"
Sep 17 14:25:50.437890 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:50.437836    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/341bef6c-9dd7-4f4c-b1cc-dc69e421ab98-dshm\") pod \"noten-model-0c1110-kserve-5797796699-d6ssx\" (UID: \"341bef6c-9dd7-4f4c-b1cc-dc69e421ab98\") " pod="ai-tenant-e2e-noten-0c1110/noten-model-0c1110-kserve-5797796699-d6ssx"
Sep 17 14:25:50.438096 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:50.437914    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtpqn\" (UniqueName: \"kubernetes.io/projected/341bef6c-9dd7-4f4c-b1cc-dc69e421ab98-kube-api-access-rtpqn\") pod \"noten-model-0c1110-kserve-5797796699-d6ssx\" (UID: \"341bef6c-9dd7-4f4c-b1cc-dc69e421ab98\") " pod="ai-tenant-e2e-noten-0c1110/noten-model-0c1110-kserve-5797796699-d6ssx"
Sep 17 14:25:50.438096 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:50.437989    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/341bef6c-9dd7-4f4c-b1cc-dc69e421ab98-tmp-dir\") pod \"noten-model-0c1110-kserve-5797796699-d6ssx\" (UID: \"341bef6c-9dd7-4f4c-b1cc-dc69e421ab98\") " pod="ai-tenant-e2e-noten-0c1110/noten-model-0c1110-kserve-5797796699-d6ssx"
Sep 17 14:25:50.438228 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:50.438174    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/341bef6c-9dd7-4f4c-b1cc-dc69e421ab98-home\") pod \"noten-model-0c1110-kserve-5797796699-d6ssx\" (UID: \"341bef6c-9dd7-4f4c-b1cc-dc69e421ab98\") " pod="ai-tenant-e2e-noten-0c1110/noten-model-0c1110-kserve-5797796699-d6ssx"
Sep 17 14:25:50.438344 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:50.438322    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/341bef6c-9dd7-4f4c-b1cc-dc69e421ab98-model-cache\") pod \"noten-model-0c1110-kserve-5797796699-d6ssx\" (UID: \"341bef6c-9dd7-4f4c-b1cc-dc69e421ab98\") " pod="ai-tenant-e2e-noten-0c1110/noten-model-0c1110-kserve-5797796699-d6ssx"
Sep 17 14:25:50.438514 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:50.438357    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/341bef6c-9dd7-4f4c-b1cc-dc69e421ab98-tmp-dir\") pod \"noten-model-0c1110-kserve-5797796699-d6ssx\" (UID: \"341bef6c-9dd7-4f4c-b1cc-dc69e421ab98\") " pod="ai-tenant-e2e-noten-0c1110/noten-model-0c1110-kserve-5797796699-d6ssx"
Sep 17 14:25:50.440193 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:50.440173    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/341bef6c-9dd7-4f4c-b1cc-dc69e421ab98-dshm\") pod \"noten-model-0c1110-kserve-5797796699-d6ssx\" (UID: \"341bef6c-9dd7-4f4c-b1cc-dc69e421ab98\") " pod="ai-tenant-e2e-noten-0c1110/noten-model-0c1110-kserve-5797796699-d6ssx"
Sep 17 14:25:50.440417 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:50.440397    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/341bef6c-9dd7-4f4c-b1cc-dc69e421ab98-tls-certs\") pod \"noten-model-0c1110-kserve-5797796699-d6ssx\" (UID: \"341bef6c-9dd7-4f4c-b1cc-dc69e421ab98\") " pod="ai-tenant-e2e-noten-0c1110/noten-model-0c1110-kserve-5797796699-d6ssx"
Sep 17 14:25:50.446456 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:50.446432    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtpqn\" (UniqueName: \"kubernetes.io/projected/341bef6c-9dd7-4f4c-b1cc-dc69e421ab98-kube-api-access-rtpqn\") pod \"noten-model-0c1110-kserve-5797796699-d6ssx\" (UID: \"341bef6c-9dd7-4f4c-b1cc-dc69e421ab98\") " pod="ai-tenant-e2e-noten-0c1110/noten-model-0c1110-kserve-5797796699-d6ssx"
Sep 17 14:25:50.620089 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:50.620003    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-0c1110/noten-model-0c1110-kserve-5797796699-d6ssx"
Sep 17 14:25:50.912973 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:50.912944    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-0c1110/noten-model-0c1110-kserve-5797796699-d6ssx"]
Sep 17 14:25:50.915113 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:25:50.915085    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod341bef6c_9dd7_4f4c_b1cc_dc69e421ab98.slice/crio-b626dccfffd31f994641915c78cc7a363d2c336a19c3301aad4ece839682008f WatchSource:0}: Error finding container b626dccfffd31f994641915c78cc7a363d2c336a19c3301aad4ece839682008f: Status 404 returned error can't find the container with id b626dccfffd31f994641915c78cc7a363d2c336a19c3301aad4ece839682008f
Sep 17 14:25:51.640749 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:51.640712    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-0c1110/noten-model-0c1110-kserve-5797796699-d6ssx" event={"ID":"341bef6c-9dd7-4f4c-b1cc-dc69e421ab98","Type":"ContainerStarted","Data":"3f94109a0eaa3e7b9131334f8ee30f322f1298a2465b86abe162f8a874e5182b"}
Sep 17 14:25:51.641126 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:51.640756    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-0c1110/noten-model-0c1110-kserve-5797796699-d6ssx" event={"ID":"341bef6c-9dd7-4f4c-b1cc-dc69e421ab98","Type":"ContainerStarted","Data":"b626dccfffd31f994641915c78cc7a363d2c336a19c3301aad4ece839682008f"}
Sep 17 14:25:51.663385 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:51.663333    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-0c1110/noten-model-0c1110-kserve-5797796699-d6ssx" podStartSLOduration=1.663313526 podStartE2EDuration="1.663313526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:25:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:25:51.661536159 +0000 UTC m=+1252.663079636" watchObservedRunningTime="2026-09-17 14:25:51.663313526 +0000 UTC m=+1252.664857001"
Sep 17 14:25:53.806128 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:53.800146    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-181b16-tpknt"]
Sep 17 14:25:53.806128 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:53.804940    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-181b16-tpknt"
Sep 17 14:25:53.807914 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:53.807893    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ddwrn\""
Sep 17 14:25:53.815100 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:53.814692    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-181b16-tpknt"]
Sep 17 14:25:53.874196 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:53.874161    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f41287c-8dcd-419e-bd20-38a16fd5b66c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-181b16-tpknt\" (UID: \"2f41287c-8dcd-419e-bd20-38a16fd5b66c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-181b16-tpknt"
Sep 17 14:25:53.975793 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:53.975753    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f41287c-8dcd-419e-bd20-38a16fd5b66c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-181b16-tpknt\" (UID: \"2f41287c-8dcd-419e-bd20-38a16fd5b66c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-181b16-tpknt"
Sep 17 14:25:53.976406 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:53.976383    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f41287c-8dcd-419e-bd20-38a16fd5b66c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-181b16-tpknt\" (UID: \"2f41287c-8dcd-419e-bd20-38a16fd5b66c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-181b16-tpknt"
Sep 17 14:25:54.125101 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:54.125019    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-181b16-tpknt"
Sep 17 14:25:54.175574 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:54.175330    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:25:54.411449 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:54.411415    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-181b16-tpknt"]
Sep 17 14:25:54.413174 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:25:54.413142    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f41287c_8dcd_419e_bd20_38a16fd5b66c.slice/crio-88b34a638863236168780aea9cb787a5e36f24c0bd29d6183010086320890395 WatchSource:0}: Error finding container 88b34a638863236168780aea9cb787a5e36f24c0bd29d6183010086320890395: Status 404 returned error can't find the container with id 88b34a638863236168780aea9cb787a5e36f24c0bd29d6183010086320890395
Sep 17 14:25:54.660299 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:54.660219    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-181b16-tpknt" event={"ID":"2f41287c-8dcd-419e-bd20-38a16fd5b66c","Type":"ContainerStarted","Data":"88b34a638863236168780aea9cb787a5e36f24c0bd29d6183010086320890395"}
Sep 17 14:25:54.950938 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:54.950893    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:25:55.668825 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:55.668733    2582 generic.go:356] "Generic (PLEG): container finished" podID="2f41287c-8dcd-419e-bd20-38a16fd5b66c" containerID="ca986e3a65b2ef269b6dc064ee37ab78bbb389eacd8c66ecf75e127ec9e74ad7" exitCode=0
Sep 17 14:25:55.668825 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:55.668776    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-181b16-tpknt" event={"ID":"2f41287c-8dcd-419e-bd20-38a16fd5b66c","Type":"ContainerDied","Data":"ca986e3a65b2ef269b6dc064ee37ab78bbb389eacd8c66ecf75e127ec9e74ad7"}
Sep 17 14:25:55.741525 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:55.741489    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:25:56.446155 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:56.446112    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:25:56.868228 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:56.868199    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-181b16-tpknt"
Sep 17 14:25:56.906357 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:56.906321    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f41287c-8dcd-419e-bd20-38a16fd5b66c-maas-api-service-ca\") pod \"2f41287c-8dcd-419e-bd20-38a16fd5b66c\" (UID: \"2f41287c-8dcd-419e-bd20-38a16fd5b66c\") "
Sep 17 14:25:56.906746 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:56.906720    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f41287c-8dcd-419e-bd20-38a16fd5b66c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2f41287c-8dcd-419e-bd20-38a16fd5b66c" (UID: "2f41287c-8dcd-419e-bd20-38a16fd5b66c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:25:56.906875 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:56.906850    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f41287c-8dcd-419e-bd20-38a16fd5b66c-maas-api-service-ca\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:25:57.243623 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:57.243554    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:25:57.682990 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:57.682960    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-181b16-tpknt"
Sep 17 14:25:57.682990 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:57.682987    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-181b16-tpknt" event={"ID":"2f41287c-8dcd-419e-bd20-38a16fd5b66c","Type":"ContainerDied","Data":"88b34a638863236168780aea9cb787a5e36f24c0bd29d6183010086320890395"}
Sep 17 14:25:57.683459 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:57.683031    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="88b34a638863236168780aea9cb787a5e36f24c0bd29d6183010086320890395"
Sep 17 14:25:57.876596 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:57.876484    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:25:58.657948 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:58.657915    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:25:59.340454 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:59.340415    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:25:59.568010 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:25:59.567979    2582 scope.go:117] "RemoveContainer" containerID="5fcb719a644bd0018bcc09a950d608bf1727bf85439e709b466595b53d4c43e3"
Sep 17 14:26:00.133178 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:00.133133    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:26:00.620888 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:00.620850    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-0c1110/noten-model-0c1110-kserve-5797796699-d6ssx"
Sep 17 14:26:00.621761 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:00.621735    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-noten-0c1110/noten-model-0c1110-kserve-5797796699-d6ssx"
Sep 17 14:26:00.663991 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:00.663962    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-noten-0c1110/noten-model-0c1110-kserve-5797796699-d6ssx"
Sep 17 14:26:00.692534 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:00.692491    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:26:00.715495 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:00.715467    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-noten-0c1110/noten-model-0c1110-kserve-5797796699-d6ssx"
Sep 17 14:26:01.403495 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:01.403462    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:26:01.593320 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:01.593258    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:26:02.254631 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:02.254588    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:26:03.179746 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:03.179710    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:26:04.048283 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:04.048246    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:26:04.509833 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:04.509797    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:26:04.539879 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:04.539843    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-181b16-bd664686c-hq5fv"]
Sep 17 14:26:04.540454 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:04.540416    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-181b16-bd664686c-hq5fv" podUID="e285562e-f8c3-45a2-b537-aae94a204392" containerName="payload-processing" containerID="cri-o://ab17cb81e5eabc547b8971ff3eb6f32bb85b2f2539d5bdc356d0f13792d3988d" gracePeriod=30
Sep 17 14:26:04.540906 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:04.540884    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-181b16-55c996cb6-prk6x"]
Sep 17 14:26:04.541215 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:04.541193    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-181b16-55c996cb6-prk6x" podUID="72dead9f-9eb2-4ac2-a8af-a85ff63ee44f" containerName="payload-pre-processing" containerID="cri-o://0d4759687bb68a5ac51648f6a3a4a09e173cb98300f2973a7d887f6cbaf734cc" gracePeriod=30
Sep 17 14:26:04.635481 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:04.635436    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:26:04.689810 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:04.689756    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-181b16-tpknt"]
Sep 17 14:26:04.693904 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:04.693855    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-181b16-tpknt"]
Sep 17 14:26:04.722307 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:04.722278    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:26:04.729882 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:04.729850    2582 generic.go:356] "Generic (PLEG): container finished" podID="72dead9f-9eb2-4ac2-a8af-a85ff63ee44f" containerID="0d4759687bb68a5ac51648f6a3a4a09e173cb98300f2973a7d887f6cbaf734cc" exitCode=0
Sep 17 14:26:04.730087 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:04.730066    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-181b16-55c996cb6-prk6x" event={"ID":"72dead9f-9eb2-4ac2-a8af-a85ff63ee44f","Type":"ContainerDied","Data":"0d4759687bb68a5ac51648f6a3a4a09e173cb98300f2973a7d887f6cbaf734cc"}
Sep 17 14:26:04.732183 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:04.732043    2582 generic.go:356] "Generic (PLEG): container finished" podID="e285562e-f8c3-45a2-b537-aae94a204392" containerID="ab17cb81e5eabc547b8971ff3eb6f32bb85b2f2539d5bdc356d0f13792d3988d" exitCode=0
Sep 17 14:26:04.732183 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:04.732129    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-181b16-bd664686c-hq5fv" event={"ID":"e285562e-f8c3-45a2-b537-aae94a204392","Type":"ContainerDied","Data":"ab17cb81e5eabc547b8971ff3eb6f32bb85b2f2539d5bdc356d0f13792d3988d"}
Sep 17 14:26:04.920890 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:04.920855    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-181b16-55c996cb6-prk6x"
Sep 17 14:26:04.924067 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:04.924041    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-181b16-bd664686c-hq5fv"
Sep 17 14:26:05.010987 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:05.010948    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e285562e-f8c3-45a2-b537-aae94a204392-plugins-config-volume\") pod \"e285562e-f8c3-45a2-b537-aae94a204392\" (UID: \"e285562e-f8c3-45a2-b537-aae94a204392\") "
Sep 17 14:26:05.011141 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:05.011027    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tngg4\" (UniqueName: \"kubernetes.io/projected/72dead9f-9eb2-4ac2-a8af-a85ff63ee44f-kube-api-access-tngg4\") pod \"72dead9f-9eb2-4ac2-a8af-a85ff63ee44f\" (UID: \"72dead9f-9eb2-4ac2-a8af-a85ff63ee44f\") "
Sep 17 14:26:05.011141 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:05.011092    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72dead9f-9eb2-4ac2-a8af-a85ff63ee44f-plugins-config-volume\") pod \"72dead9f-9eb2-4ac2-a8af-a85ff63ee44f\" (UID: \"72dead9f-9eb2-4ac2-a8af-a85ff63ee44f\") "
Sep 17 14:26:05.011141 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:05.011131    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lpqjg\" (UniqueName: \"kubernetes.io/projected/e285562e-f8c3-45a2-b537-aae94a204392-kube-api-access-lpqjg\") pod \"e285562e-f8c3-45a2-b537-aae94a204392\" (UID: \"e285562e-f8c3-45a2-b537-aae94a204392\") "
Sep 17 14:26:05.011433 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:05.011408    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e285562e-f8c3-45a2-b537-aae94a204392-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e285562e-f8c3-45a2-b537-aae94a204392" (UID: "e285562e-f8c3-45a2-b537-aae94a204392"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:26:05.011510 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:05.011457    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72dead9f-9eb2-4ac2-a8af-a85ff63ee44f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "72dead9f-9eb2-4ac2-a8af-a85ff63ee44f" (UID: "72dead9f-9eb2-4ac2-a8af-a85ff63ee44f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:26:05.011588 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:05.011571    2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e285562e-f8c3-45a2-b537-aae94a204392-plugins-config-volume\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:26:05.011656 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:05.011591    2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72dead9f-9eb2-4ac2-a8af-a85ff63ee44f-plugins-config-volume\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:26:05.013348 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:05.013327    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e285562e-f8c3-45a2-b537-aae94a204392-kube-api-access-lpqjg" (OuterVolumeSpecName: "kube-api-access-lpqjg") pod "e285562e-f8c3-45a2-b537-aae94a204392" (UID: "e285562e-f8c3-45a2-b537-aae94a204392"). InnerVolumeSpecName "kube-api-access-lpqjg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:26:05.013410 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:05.013357    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72dead9f-9eb2-4ac2-a8af-a85ff63ee44f-kube-api-access-tngg4" (OuterVolumeSpecName: "kube-api-access-tngg4") pod "72dead9f-9eb2-4ac2-a8af-a85ff63ee44f" (UID: "72dead9f-9eb2-4ac2-a8af-a85ff63ee44f"). InnerVolumeSpecName "kube-api-access-tngg4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:26:05.055536 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:05.055500    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:26:05.113122 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:05.113027    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tngg4\" (UniqueName: \"kubernetes.io/projected/72dead9f-9eb2-4ac2-a8af-a85ff63ee44f-kube-api-access-tngg4\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:26:05.113122 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:05.113065    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lpqjg\" (UniqueName: \"kubernetes.io/projected/e285562e-f8c3-45a2-b537-aae94a204392-kube-api-access-lpqjg\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:26:05.225822 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:05.225770    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:26:05.496195 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:05.496161    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f41287c-8dcd-419e-bd20-38a16fd5b66c" path="/var/lib/kubelet/pods/2f41287c-8dcd-419e-bd20-38a16fd5b66c/volumes"
Sep 17 14:26:05.739665 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:05.739560    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-181b16-bd664686c-hq5fv"
Sep 17 14:26:05.739665 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:05.739586    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-181b16-bd664686c-hq5fv" event={"ID":"e285562e-f8c3-45a2-b537-aae94a204392","Type":"ContainerDied","Data":"c4495959ed79c434bfd80bed640fd9d050cee7242cbb995dc418c8dc4fe79cfb"}
Sep 17 14:26:05.739903 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:05.739671    2582 scope.go:117] "RemoveContainer" containerID="ab17cb81e5eabc547b8971ff3eb6f32bb85b2f2539d5bdc356d0f13792d3988d"
Sep 17 14:26:05.741670 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:05.741307    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-181b16-55c996cb6-prk6x"
Sep 17 14:26:05.741670 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:05.741501    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-181b16-55c996cb6-prk6x" event={"ID":"72dead9f-9eb2-4ac2-a8af-a85ff63ee44f","Type":"ContainerDied","Data":"3e7605605a56ac3276f7faf8d0df9d6e63ecc4e85b6a1ffada5fb846cff50c1d"}
Sep 17 14:26:05.744046 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:05.744018    2582 generic.go:356] "Generic (PLEG): container finished" podID="3ab0e2f8-287d-4ba7-b06c-e3bef8457239" containerID="015b660060515c2ef0e8448611ac637a732cd9c330dd99d69e03445a60aab678" exitCode=0
Sep 17 14:26:05.744192 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:05.744151    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn" event={"ID":"3ab0e2f8-287d-4ba7-b06c-e3bef8457239","Type":"ContainerDied","Data":"015b660060515c2ef0e8448611ac637a732cd9c330dd99d69e03445a60aab678"}
Sep 17 14:26:05.744699 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:05.744682    2582 scope.go:117] "RemoveContainer" containerID="015b660060515c2ef0e8448611ac637a732cd9c330dd99d69e03445a60aab678"
Sep 17 14:26:05.745939 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:05.745903    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:26:05.746034 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:05.745971    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:26:05.746034 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:05.746004    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:26:05.752751 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:05.752736    2582 scope.go:117] "RemoveContainer" containerID="0d4759687bb68a5ac51648f6a3a4a09e173cb98300f2973a7d887f6cbaf734cc"
Sep 17 14:26:05.787805 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:05.787771    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-181b16-bd664686c-hq5fv"]
Sep 17 14:26:05.797689 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:05.797657    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-181b16-bd664686c-hq5fv"]
Sep 17 14:26:05.809433 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:05.809377    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-181b16-55c996cb6-prk6x"]
Sep 17 14:26:05.812311 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:05.812282    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-181b16-55c996cb6-prk6x"]
Sep 17 14:26:05.910713 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:05.910673    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:26:06.752163 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:06.752074    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:26:06.758130 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:06.758098    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn" event={"ID":"3ab0e2f8-287d-4ba7-b06c-e3bef8457239","Type":"ContainerStarted","Data":"924ff6cafa7b48480c3d85b6ec4e75b58a864e0b94bbd54637650dc887286726"}
Sep 17 14:26:07.255763 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:07.255707    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn"
Sep 17 14:26:07.255991 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:07.255835    2582 patch_prober.go:28] interesting pod/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Sep 17 14:26:07.255991 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:07.255921    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn" podUID="3ab0e2f8-287d-4ba7-b06c-e3bef8457239" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Sep 17 14:26:07.441510 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:07.441458    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:26:07.500261 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:07.500218    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72dead9f-9eb2-4ac2-a8af-a85ff63ee44f" path="/var/lib/kubelet/pods/72dead9f-9eb2-4ac2-a8af-a85ff63ee44f/volumes"
Sep 17 14:26:07.501066 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:07.501032    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e285562e-f8c3-45a2-b537-aae94a204392" path="/var/lib/kubelet/pods/e285562e-f8c3-45a2-b537-aae94a204392/volumes"
Sep 17 14:26:08.255772 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:08.255729    2582 patch_prober.go:28] interesting pod/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Sep 17 14:26:08.256262 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:08.255794    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn" podUID="3ab0e2f8-287d-4ba7-b06c-e3bef8457239" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Sep 17 14:26:08.339174 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:08.339078    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:26:09.205369 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:09.205331    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:26:09.255821 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:09.255781    2582 patch_prober.go:28] interesting pod/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Sep 17 14:26:09.256223 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:09.255849    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn" podUID="3ab0e2f8-287d-4ba7-b06c-e3bef8457239" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Sep 17 14:26:09.725034 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:09.724996    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:26:10.136969 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:10.136860    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:26:10.255865 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:10.255818    2582 patch_prober.go:28] interesting pod/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Sep 17 14:26:10.256416 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:10.255917    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn" podUID="3ab0e2f8-287d-4ba7-b06c-e3bef8457239" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Sep 17 14:26:11.255319 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:11.255274    2582 patch_prober.go:28] interesting pod/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Sep 17 14:26:11.255536 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:11.255364    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn" podUID="3ab0e2f8-287d-4ba7-b06c-e3bef8457239" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Sep 17 14:26:12.259898 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:12.259865    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn"
Sep 17 14:26:12.260336 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:12.260156    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn"
Sep 17 14:26:12.260979 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:12.260959    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn"
Sep 17 14:26:23.254750 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:23.254712    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6"]
Sep 17 14:26:23.255219 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:23.254912    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6" podUID="43bc21dc-416c-42bd-8176-2cf465dd544e" containerName="istio-proxy" containerID="cri-o://b8646fd9c22472cb05d857e49e036cd5e3e6e915d13a30952cd737c434b8bd9b" gracePeriod=30
Sep 17 14:26:23.284317 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:23.284280    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:26:23.522073 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:23.521969    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:26:23.643567 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:23.643529    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:26:23.956085 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:23.956030    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:26:25.819485 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:25.819446    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-9b8115/test-model-9b8115-kserve-7896bbcf7c-x294s"]
Sep 17 14:26:25.841874 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:25.841828    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:26:28.013706 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:28.013668    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-0c1110/noten-model-0c1110-kserve-5797796699-d6ssx"]
Sep 17 14:26:28.061726 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:28.061694    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:26:28.522254 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:28.522183    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6"
Sep 17 14:26:28.581756 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:28.581670    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43bc21dc-416c-42bd-8176-2cf465dd544e-istio-envoy\") pod \"43bc21dc-416c-42bd-8176-2cf465dd544e\" (UID: \"43bc21dc-416c-42bd-8176-2cf465dd544e\") "
Sep 17 14:26:28.581950 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:28.581814    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43bc21dc-416c-42bd-8176-2cf465dd544e-istiod-ca-cert\") pod \"43bc21dc-416c-42bd-8176-2cf465dd544e\" (UID: \"43bc21dc-416c-42bd-8176-2cf465dd544e\") "
Sep 17 14:26:28.581950 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:28.581861    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43bc21dc-416c-42bd-8176-2cf465dd544e-workload-socket\") pod \"43bc21dc-416c-42bd-8176-2cf465dd544e\" (UID: \"43bc21dc-416c-42bd-8176-2cf465dd544e\") "
Sep 17 14:26:28.581950 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:28.581889    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43bc21dc-416c-42bd-8176-2cf465dd544e-workload-certs\") pod \"43bc21dc-416c-42bd-8176-2cf465dd544e\" (UID: \"43bc21dc-416c-42bd-8176-2cf465dd544e\") "
Sep 17 14:26:28.581950 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:28.581921    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43bc21dc-416c-42bd-8176-2cf465dd544e-istio-token\") pod \"43bc21dc-416c-42bd-8176-2cf465dd544e\" (UID: \"43bc21dc-416c-42bd-8176-2cf465dd544e\") "
Sep 17 14:26:28.582179 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:28.581965    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43bc21dc-416c-42bd-8176-2cf465dd544e-istio-data\") pod \"43bc21dc-416c-42bd-8176-2cf465dd544e\" (UID: \"43bc21dc-416c-42bd-8176-2cf465dd544e\") "
Sep 17 14:26:28.582179 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:28.582044    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43bc21dc-416c-42bd-8176-2cf465dd544e-istio-podinfo\") pod \"43bc21dc-416c-42bd-8176-2cf465dd544e\" (UID: \"43bc21dc-416c-42bd-8176-2cf465dd544e\") "
Sep 17 14:26:28.582179 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:28.582114    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43bc21dc-416c-42bd-8176-2cf465dd544e-credential-socket\") pod \"43bc21dc-416c-42bd-8176-2cf465dd544e\" (UID: \"43bc21dc-416c-42bd-8176-2cf465dd544e\") "
Sep 17 14:26:28.582328 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:28.582181    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fkj4s\" (UniqueName: \"kubernetes.io/projected/43bc21dc-416c-42bd-8176-2cf465dd544e-kube-api-access-fkj4s\") pod \"43bc21dc-416c-42bd-8176-2cf465dd544e\" (UID: \"43bc21dc-416c-42bd-8176-2cf465dd544e\") "
Sep 17 14:26:28.582328 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:28.582213    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43bc21dc-416c-42bd-8176-2cf465dd544e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "43bc21dc-416c-42bd-8176-2cf465dd544e" (UID: "43bc21dc-416c-42bd-8176-2cf465dd544e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:26:28.582426 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:28.582331    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43bc21dc-416c-42bd-8176-2cf465dd544e-istio-data" (OuterVolumeSpecName: "istio-data") pod "43bc21dc-416c-42bd-8176-2cf465dd544e" (UID: "43bc21dc-416c-42bd-8176-2cf465dd544e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:26:28.582426 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:28.582400    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43bc21dc-416c-42bd-8176-2cf465dd544e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "43bc21dc-416c-42bd-8176-2cf465dd544e" (UID: "43bc21dc-416c-42bd-8176-2cf465dd544e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:26:28.582541 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:28.582435    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43bc21dc-416c-42bd-8176-2cf465dd544e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "43bc21dc-416c-42bd-8176-2cf465dd544e" (UID: "43bc21dc-416c-42bd-8176-2cf465dd544e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:26:28.582676 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:28.582650    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43bc21dc-416c-42bd-8176-2cf465dd544e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "43bc21dc-416c-42bd-8176-2cf465dd544e" (UID: "43bc21dc-416c-42bd-8176-2cf465dd544e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:26:28.582847 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:28.582716    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43bc21dc-416c-42bd-8176-2cf465dd544e-workload-socket\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:26:28.582939 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:28.582880    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43bc21dc-416c-42bd-8176-2cf465dd544e-workload-certs\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:26:28.582939 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:28.582898    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43bc21dc-416c-42bd-8176-2cf465dd544e-istio-data\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:26:28.582939 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:28.582913    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43bc21dc-416c-42bd-8176-2cf465dd544e-credential-socket\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:26:28.584667 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:28.584584    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43bc21dc-416c-42bd-8176-2cf465dd544e-istio-token" (OuterVolumeSpecName: "istio-token") pod "43bc21dc-416c-42bd-8176-2cf465dd544e" (UID: "43bc21dc-416c-42bd-8176-2cf465dd544e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:26:28.584667 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:28.584651    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43bc21dc-416c-42bd-8176-2cf465dd544e-kube-api-access-fkj4s" (OuterVolumeSpecName: "kube-api-access-fkj4s") pod "43bc21dc-416c-42bd-8176-2cf465dd544e" (UID: "43bc21dc-416c-42bd-8176-2cf465dd544e"). InnerVolumeSpecName "kube-api-access-fkj4s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:26:28.584847 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:28.584773    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43bc21dc-416c-42bd-8176-2cf465dd544e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "43bc21dc-416c-42bd-8176-2cf465dd544e" (UID: "43bc21dc-416c-42bd-8176-2cf465dd544e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:26:28.584847 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:28.584784    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/43bc21dc-416c-42bd-8176-2cf465dd544e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "43bc21dc-416c-42bd-8176-2cf465dd544e" (UID: "43bc21dc-416c-42bd-8176-2cf465dd544e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 14:26:28.683995 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:28.683956    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fkj4s\" (UniqueName: \"kubernetes.io/projected/43bc21dc-416c-42bd-8176-2cf465dd544e-kube-api-access-fkj4s\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:26:28.683995 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:28.683993    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43bc21dc-416c-42bd-8176-2cf465dd544e-istio-envoy\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:26:28.683995 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:28.684003    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43bc21dc-416c-42bd-8176-2cf465dd544e-istiod-ca-cert\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:26:28.684245 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:28.684012    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43bc21dc-416c-42bd-8176-2cf465dd544e-istio-token\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:26:28.684245 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:28.684020    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43bc21dc-416c-42bd-8176-2cf465dd544e-istio-podinfo\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:26:28.718516 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:28.718481    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:26:28.918941 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:28.918848    2582 generic.go:356] "Generic (PLEG): container finished" podID="43bc21dc-416c-42bd-8176-2cf465dd544e" containerID="b8646fd9c22472cb05d857e49e036cd5e3e6e915d13a30952cd737c434b8bd9b" exitCode=0
Sep 17 14:26:28.918941 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:28.918899    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6" event={"ID":"43bc21dc-416c-42bd-8176-2cf465dd544e","Type":"ContainerDied","Data":"b8646fd9c22472cb05d857e49e036cd5e3e6e915d13a30952cd737c434b8bd9b"}
Sep 17 14:26:28.919120 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:28.918962    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6" event={"ID":"43bc21dc-416c-42bd-8176-2cf465dd544e","Type":"ContainerDied","Data":"99b6e96335c9d2da6e39ce69ebf9f183094864fc880449fbdcd14fddc37960d5"}
Sep 17 14:26:28.919120 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:28.918901    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6"
Sep 17 14:26:28.919120 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:28.918992    2582 scope.go:117] "RemoveContainer" containerID="b8646fd9c22472cb05d857e49e036cd5e3e6e915d13a30952cd737c434b8bd9b"
Sep 17 14:26:28.931335 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:28.931306    2582 scope.go:117] "RemoveContainer" containerID="b8646fd9c22472cb05d857e49e036cd5e3e6e915d13a30952cd737c434b8bd9b"
Sep 17 14:26:28.931689 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:26:28.931662    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8646fd9c22472cb05d857e49e036cd5e3e6e915d13a30952cd737c434b8bd9b\": container with ID starting with b8646fd9c22472cb05d857e49e036cd5e3e6e915d13a30952cd737c434b8bd9b not found: ID does not exist" containerID="b8646fd9c22472cb05d857e49e036cd5e3e6e915d13a30952cd737c434b8bd9b"
Sep 17 14:26:28.931785 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:28.931700    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8646fd9c22472cb05d857e49e036cd5e3e6e915d13a30952cd737c434b8bd9b"} err="failed to get container status \"b8646fd9c22472cb05d857e49e036cd5e3e6e915d13a30952cd737c434b8bd9b\": rpc error: code = NotFound desc = could not find container \"b8646fd9c22472cb05d857e49e036cd5e3e6e915d13a30952cd737c434b8bd9b\": container with ID starting with b8646fd9c22472cb05d857e49e036cd5e3e6e915d13a30952cd737c434b8bd9b not found: ID does not exist"
Sep 17 14:26:28.943118 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:28.943089    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6"]
Sep 17 14:26:28.946755 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:28.946727    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-0c1110-openshift-default-58659488d4-t86f6"]
Sep 17 14:26:29.494784 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:29.494752    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43bc21dc-416c-42bd-8176-2cf465dd544e" path="/var/lib/kubelet/pods/43bc21dc-416c-42bd-8176-2cf465dd544e/volumes"
Sep 17 14:26:37.877645 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:37.877592    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-1a095586/e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7"]
Sep 17 14:26:37.878209 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:37.878188    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f41287c-8dcd-419e-bd20-38a16fd5b66c" containerName="revoke-keys"
Sep 17 14:26:37.878288 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:37.878211    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f41287c-8dcd-419e-bd20-38a16fd5b66c" containerName="revoke-keys"
Sep 17 14:26:37.878288 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:37.878240    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e285562e-f8c3-45a2-b537-aae94a204392" containerName="payload-processing"
Sep 17 14:26:37.878288 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:37.878249    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e285562e-f8c3-45a2-b537-aae94a204392" containerName="payload-processing"
Sep 17 14:26:37.878288 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:37.878268    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43bc21dc-416c-42bd-8176-2cf465dd544e" containerName="istio-proxy"
Sep 17 14:26:37.878288 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:37.878276    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="43bc21dc-416c-42bd-8176-2cf465dd544e" containerName="istio-proxy"
Sep 17 14:26:37.878559 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:37.878293    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72dead9f-9eb2-4ac2-a8af-a85ff63ee44f" containerName="payload-pre-processing"
Sep 17 14:26:37.878559 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:37.878303    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="72dead9f-9eb2-4ac2-a8af-a85ff63ee44f" containerName="payload-pre-processing"
Sep 17 14:26:37.878559 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:37.878424    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e285562e-f8c3-45a2-b537-aae94a204392" containerName="payload-processing"
Sep 17 14:26:37.878559 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:37.878435    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="43bc21dc-416c-42bd-8176-2cf465dd544e" containerName="istio-proxy"
Sep 17 14:26:37.878559 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:37.878447    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f41287c-8dcd-419e-bd20-38a16fd5b66c" containerName="revoke-keys"
Sep 17 14:26:37.878559 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:37.878455    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="72dead9f-9eb2-4ac2-a8af-a85ff63ee44f" containerName="payload-pre-processing"
Sep 17 14:26:37.881438 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:37.881418    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-1a095586/e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7"
Sep 17 14:26:37.884061 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:37.884043    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-1a095586\"/\"kube-root-ca.crt\""
Sep 17 14:26:37.884940 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:37.884923    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-1a095586\"/\"default-dockercfg-bmtg7\""
Sep 17 14:26:37.885237 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:37.885223    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-1a095586\"/\"openshift-service-ca.crt\""
Sep 17 14:26:37.885324 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:37.885303    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-1a095586\"/\"e2e-lifecycle-model-1a095586-kserve-self-signed-certs\""
Sep 17 14:26:37.895181 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:37.895156    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-1a095586/e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7"]
Sep 17 14:26:37.928134 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:37.928101    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:26:37.979209 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:37.979179    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/182c2e69-c5c8-41e8-8190-a8bc6ca56cf2-dshm\") pod \"e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7\" (UID: \"182c2e69-c5c8-41e8-8190-a8bc6ca56cf2\") " pod="ai-tenant-e2e-mt-1a095586/e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7"
Sep 17 14:26:37.979329 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:37.979231    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gn49s\" (UniqueName: \"kubernetes.io/projected/182c2e69-c5c8-41e8-8190-a8bc6ca56cf2-kube-api-access-gn49s\") pod \"e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7\" (UID: \"182c2e69-c5c8-41e8-8190-a8bc6ca56cf2\") " pod="ai-tenant-e2e-mt-1a095586/e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7"
Sep 17 14:26:37.979329 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:37.979251    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/182c2e69-c5c8-41e8-8190-a8bc6ca56cf2-model-cache\") pod \"e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7\" (UID: \"182c2e69-c5c8-41e8-8190-a8bc6ca56cf2\") " pod="ai-tenant-e2e-mt-1a095586/e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7"
Sep 17 14:26:37.979329 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:37.979285    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/182c2e69-c5c8-41e8-8190-a8bc6ca56cf2-tmp-dir\") pod \"e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7\" (UID: \"182c2e69-c5c8-41e8-8190-a8bc6ca56cf2\") " pod="ai-tenant-e2e-mt-1a095586/e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7"
Sep 17 14:26:37.979451 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:37.979353    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/182c2e69-c5c8-41e8-8190-a8bc6ca56cf2-home\") pod \"e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7\" (UID: \"182c2e69-c5c8-41e8-8190-a8bc6ca56cf2\") " pod="ai-tenant-e2e-mt-1a095586/e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7"
Sep 17 14:26:37.979486 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:37.979468    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/182c2e69-c5c8-41e8-8190-a8bc6ca56cf2-tls-certs\") pod \"e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7\" (UID: \"182c2e69-c5c8-41e8-8190-a8bc6ca56cf2\") " pod="ai-tenant-e2e-mt-1a095586/e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7"
Sep 17 14:26:38.080895 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:38.080847    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/182c2e69-c5c8-41e8-8190-a8bc6ca56cf2-home\") pod \"e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7\" (UID: \"182c2e69-c5c8-41e8-8190-a8bc6ca56cf2\") " pod="ai-tenant-e2e-mt-1a095586/e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7"
Sep 17 14:26:38.081153 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:38.080971    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/182c2e69-c5c8-41e8-8190-a8bc6ca56cf2-tls-certs\") pod \"e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7\" (UID: \"182c2e69-c5c8-41e8-8190-a8bc6ca56cf2\") " pod="ai-tenant-e2e-mt-1a095586/e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7"
Sep 17 14:26:38.081153 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:38.081014    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/182c2e69-c5c8-41e8-8190-a8bc6ca56cf2-dshm\") pod \"e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7\" (UID: \"182c2e69-c5c8-41e8-8190-a8bc6ca56cf2\") " pod="ai-tenant-e2e-mt-1a095586/e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7"
Sep 17 14:26:38.081153 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:38.081069    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gn49s\" (UniqueName: \"kubernetes.io/projected/182c2e69-c5c8-41e8-8190-a8bc6ca56cf2-kube-api-access-gn49s\") pod \"e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7\" (UID: \"182c2e69-c5c8-41e8-8190-a8bc6ca56cf2\") " pod="ai-tenant-e2e-mt-1a095586/e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7"
Sep 17 14:26:38.081153 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:38.081097    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/182c2e69-c5c8-41e8-8190-a8bc6ca56cf2-model-cache\") pod \"e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7\" (UID: \"182c2e69-c5c8-41e8-8190-a8bc6ca56cf2\") " pod="ai-tenant-e2e-mt-1a095586/e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7"
Sep 17 14:26:38.081153 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:38.081139    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/182c2e69-c5c8-41e8-8190-a8bc6ca56cf2-tmp-dir\") pod \"e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7\" (UID: \"182c2e69-c5c8-41e8-8190-a8bc6ca56cf2\") " pod="ai-tenant-e2e-mt-1a095586/e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7"
Sep 17 14:26:38.081666 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:38.081632    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/182c2e69-c5c8-41e8-8190-a8bc6ca56cf2-tmp-dir\") pod \"e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7\" (UID: \"182c2e69-c5c8-41e8-8190-a8bc6ca56cf2\") " pod="ai-tenant-e2e-mt-1a095586/e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7"
Sep 17 14:26:38.082320 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:38.082293    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/182c2e69-c5c8-41e8-8190-a8bc6ca56cf2-model-cache\") pod \"e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7\" (UID: \"182c2e69-c5c8-41e8-8190-a8bc6ca56cf2\") " pod="ai-tenant-e2e-mt-1a095586/e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7"
Sep 17 14:26:38.082596 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:38.082560    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/182c2e69-c5c8-41e8-8190-a8bc6ca56cf2-home\") pod \"e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7\" (UID: \"182c2e69-c5c8-41e8-8190-a8bc6ca56cf2\") " pod="ai-tenant-e2e-mt-1a095586/e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7"
Sep 17 14:26:38.085898 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:38.084836    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/182c2e69-c5c8-41e8-8190-a8bc6ca56cf2-dshm\") pod \"e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7\" (UID: \"182c2e69-c5c8-41e8-8190-a8bc6ca56cf2\") " pod="ai-tenant-e2e-mt-1a095586/e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7"
Sep 17 14:26:38.085898 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:38.085484    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/182c2e69-c5c8-41e8-8190-a8bc6ca56cf2-tls-certs\") pod \"e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7\" (UID: \"182c2e69-c5c8-41e8-8190-a8bc6ca56cf2\") " pod="ai-tenant-e2e-mt-1a095586/e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7"
Sep 17 14:26:38.091929 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:38.091893    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gn49s\" (UniqueName: \"kubernetes.io/projected/182c2e69-c5c8-41e8-8190-a8bc6ca56cf2-kube-api-access-gn49s\") pod \"e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7\" (UID: \"182c2e69-c5c8-41e8-8190-a8bc6ca56cf2\") " pod="ai-tenant-e2e-mt-1a095586/e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7"
Sep 17 14:26:38.193798 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:38.193766    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-1a095586/e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7"
Sep 17 14:26:38.473669 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:38.473640    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-1a095586/e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7"]
Sep 17 14:26:38.476580 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:26:38.476553    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod182c2e69_c5c8_41e8_8190_a8bc6ca56cf2.slice/crio-db9ebd728d779211f5b939f2adcb6c8d7c20fdbbc8660acf438f66656a1a838f WatchSource:0}: Error finding container db9ebd728d779211f5b939f2adcb6c8d7c20fdbbc8660acf438f66656a1a838f: Status 404 returned error can't find the container with id db9ebd728d779211f5b939f2adcb6c8d7c20fdbbc8660acf438f66656a1a838f
Sep 17 14:26:38.985111 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:38.985074    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-1a095586/e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7" event={"ID":"182c2e69-c5c8-41e8-8190-a8bc6ca56cf2","Type":"ContainerStarted","Data":"eec7514c641da72913d4faf2f08ffc0f318b2fff11fbee30853586e0affad02d"}
Sep 17 14:26:38.985452 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:38.985122    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-1a095586/e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7" event={"ID":"182c2e69-c5c8-41e8-8190-a8bc6ca56cf2","Type":"ContainerStarted","Data":"db9ebd728d779211f5b939f2adcb6c8d7c20fdbbc8660acf438f66656a1a838f"}
Sep 17 14:26:39.010926 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:39.010880    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-1a095586/e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7" podStartSLOduration=2.01086548 podStartE2EDuration="2.01086548s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:26:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:26:39.010032425 +0000 UTC m=+1300.011575901" watchObservedRunningTime="2026-09-17 14:26:39.01086548 +0000 UTC m=+1300.012408997"
Sep 17 14:26:40.841397 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:40.841351    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-9b8115/test-model-9b8115-kserve-7896bbcf7c-x294s" podUID="1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb" containerName="main" containerID="cri-o://69e7e889313379cf09e7e02a8e9b78bc6ff88f1c4fdc849eddf33f6be746ffc8" gracePeriod=45
Sep 17 14:26:41.000497 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:41.000461    2582 generic.go:356] "Generic (PLEG): container finished" podID="1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb" containerID="69e7e889313379cf09e7e02a8e9b78bc6ff88f1c4fdc849eddf33f6be746ffc8" exitCode=0
Sep 17 14:26:41.000667 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:41.000538    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-9b8115/test-model-9b8115-kserve-7896bbcf7c-x294s" event={"ID":"1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb","Type":"ContainerDied","Data":"69e7e889313379cf09e7e02a8e9b78bc6ff88f1c4fdc849eddf33f6be746ffc8"}
Sep 17 14:26:41.113792 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:41.113769    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-9b8115/test-model-9b8115-kserve-7896bbcf7c-x294s"
Sep 17 14:26:41.215037 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:41.215009    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb-tmp-dir\") pod \"1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb\" (UID: \"1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb\") "
Sep 17 14:26:41.215191 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:41.215045    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-thlmh\" (UniqueName: \"kubernetes.io/projected/1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb-kube-api-access-thlmh\") pod \"1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb\" (UID: \"1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb\") "
Sep 17 14:26:41.215191 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:41.215119    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb-model-cache\") pod \"1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb\" (UID: \"1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb\") "
Sep 17 14:26:41.215285 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:41.215196    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb-tls-certs\") pod \"1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb\" (UID: \"1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb\") "
Sep 17 14:26:41.215285 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:41.215228    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb-home\") pod \"1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb\" (UID: \"1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb\") "
Sep 17 14:26:41.215399 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:41.215305    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb-dshm\") pod \"1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb\" (UID: \"1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb\") "
Sep 17 14:26:41.215456 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:41.215302    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb" (UID: "1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:26:41.215510 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:41.215485    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb-model-cache" (OuterVolumeSpecName: "model-cache") pod "1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb" (UID: "1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:26:41.215568 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:41.215522    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb-home" (OuterVolumeSpecName: "home") pod "1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb" (UID: "1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:26:41.215771 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:41.215750    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb-tmp-dir\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:26:41.215771 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:41.215771    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb-model-cache\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:26:41.215940 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:41.215785    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb-home\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:26:41.217546 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:41.217513    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb-kube-api-access-thlmh" (OuterVolumeSpecName: "kube-api-access-thlmh") pod "1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb" (UID: "1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb"). InnerVolumeSpecName "kube-api-access-thlmh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:26:41.217814 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:41.217789    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb" (UID: "1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:26:41.217867 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:41.217790    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb-dshm" (OuterVolumeSpecName: "dshm") pod "1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb" (UID: "1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:26:41.317068 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:41.317027    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb-tls-certs\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:26:41.317068 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:41.317063    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb-dshm\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:26:41.317068 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:41.317074    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-thlmh\" (UniqueName: \"kubernetes.io/projected/1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb-kube-api-access-thlmh\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:26:42.008040 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:42.008008    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-9b8115/test-model-9b8115-kserve-7896bbcf7c-x294s"
Sep 17 14:26:42.008473 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:42.008011    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-9b8115/test-model-9b8115-kserve-7896bbcf7c-x294s" event={"ID":"1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb","Type":"ContainerDied","Data":"f1a085435a66f59b90979792e7a477b47e1b34d0863189765b7f6583f0747522"}
Sep 17 14:26:42.008473 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:42.008116    2582 scope.go:117] "RemoveContainer" containerID="69e7e889313379cf09e7e02a8e9b78bc6ff88f1c4fdc849eddf33f6be746ffc8"
Sep 17 14:26:42.025427 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:42.025399    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-9b8115/test-model-9b8115-kserve-7896bbcf7c-x294s"]
Sep 17 14:26:42.029963 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:42.029934    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-9b8115/test-model-9b8115-kserve-7896bbcf7c-x294s"]
Sep 17 14:26:43.037986 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:43.037948    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-0c1110/noten-model-0c1110-kserve-5797796699-d6ssx" podUID="341bef6c-9dd7-4f4c-b1cc-dc69e421ab98" containerName="main" containerID="cri-o://3f94109a0eaa3e7b9131334f8ee30f322f1298a2465b86abe162f8a874e5182b" gracePeriod=45
Sep 17 14:26:43.112288 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:43.112254    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:26:43.309847 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:43.309824    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-0c1110/noten-model-0c1110-kserve-5797796699-d6ssx"
Sep 17 14:26:43.336623 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:43.336584    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/341bef6c-9dd7-4f4c-b1cc-dc69e421ab98-home\") pod \"341bef6c-9dd7-4f4c-b1cc-dc69e421ab98\" (UID: \"341bef6c-9dd7-4f4c-b1cc-dc69e421ab98\") "
Sep 17 14:26:43.336811 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:43.336682    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rtpqn\" (UniqueName: \"kubernetes.io/projected/341bef6c-9dd7-4f4c-b1cc-dc69e421ab98-kube-api-access-rtpqn\") pod \"341bef6c-9dd7-4f4c-b1cc-dc69e421ab98\" (UID: \"341bef6c-9dd7-4f4c-b1cc-dc69e421ab98\") "
Sep 17 14:26:43.336811 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:43.336714    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/341bef6c-9dd7-4f4c-b1cc-dc69e421ab98-tmp-dir\") pod \"341bef6c-9dd7-4f4c-b1cc-dc69e421ab98\" (UID: \"341bef6c-9dd7-4f4c-b1cc-dc69e421ab98\") "
Sep 17 14:26:43.336811 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:43.336789    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/341bef6c-9dd7-4f4c-b1cc-dc69e421ab98-model-cache\") pod \"341bef6c-9dd7-4f4c-b1cc-dc69e421ab98\" (UID: \"341bef6c-9dd7-4f4c-b1cc-dc69e421ab98\") "
Sep 17 14:26:43.336988 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:43.336870    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/341bef6c-9dd7-4f4c-b1cc-dc69e421ab98-home" (OuterVolumeSpecName: "home") pod "341bef6c-9dd7-4f4c-b1cc-dc69e421ab98" (UID: "341bef6c-9dd7-4f4c-b1cc-dc69e421ab98"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:26:43.336988 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:43.336889    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/341bef6c-9dd7-4f4c-b1cc-dc69e421ab98-dshm\") pod \"341bef6c-9dd7-4f4c-b1cc-dc69e421ab98\" (UID: \"341bef6c-9dd7-4f4c-b1cc-dc69e421ab98\") "
Sep 17 14:26:43.336988 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:43.336926    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/341bef6c-9dd7-4f4c-b1cc-dc69e421ab98-tls-certs\") pod \"341bef6c-9dd7-4f4c-b1cc-dc69e421ab98\" (UID: \"341bef6c-9dd7-4f4c-b1cc-dc69e421ab98\") "
Sep 17 14:26:43.337158 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:43.337032    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/341bef6c-9dd7-4f4c-b1cc-dc69e421ab98-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "341bef6c-9dd7-4f4c-b1cc-dc69e421ab98" (UID: "341bef6c-9dd7-4f4c-b1cc-dc69e421ab98"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:26:43.337158 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:43.337062    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/341bef6c-9dd7-4f4c-b1cc-dc69e421ab98-model-cache" (OuterVolumeSpecName: "model-cache") pod "341bef6c-9dd7-4f4c-b1cc-dc69e421ab98" (UID: "341bef6c-9dd7-4f4c-b1cc-dc69e421ab98"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:26:43.337375 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:43.337327    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/341bef6c-9dd7-4f4c-b1cc-dc69e421ab98-home\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:26:43.337375 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:43.337348    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/341bef6c-9dd7-4f4c-b1cc-dc69e421ab98-tmp-dir\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:26:43.337375 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:43.337363    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/341bef6c-9dd7-4f4c-b1cc-dc69e421ab98-model-cache\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:26:43.339583 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:43.339549    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/341bef6c-9dd7-4f4c-b1cc-dc69e421ab98-kube-api-access-rtpqn" (OuterVolumeSpecName: "kube-api-access-rtpqn") pod "341bef6c-9dd7-4f4c-b1cc-dc69e421ab98" (UID: "341bef6c-9dd7-4f4c-b1cc-dc69e421ab98"). InnerVolumeSpecName "kube-api-access-rtpqn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:26:43.339700 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:43.339578    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/341bef6c-9dd7-4f4c-b1cc-dc69e421ab98-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "341bef6c-9dd7-4f4c-b1cc-dc69e421ab98" (UID: "341bef6c-9dd7-4f4c-b1cc-dc69e421ab98"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:26:43.339815 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:43.339798    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/341bef6c-9dd7-4f4c-b1cc-dc69e421ab98-dshm" (OuterVolumeSpecName: "dshm") pod "341bef6c-9dd7-4f4c-b1cc-dc69e421ab98" (UID: "341bef6c-9dd7-4f4c-b1cc-dc69e421ab98"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:26:43.438317 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:43.438275    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/341bef6c-9dd7-4f4c-b1cc-dc69e421ab98-dshm\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:26:43.438317 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:43.438312    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/341bef6c-9dd7-4f4c-b1cc-dc69e421ab98-tls-certs\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:26:43.438317 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:43.438322    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rtpqn\" (UniqueName: \"kubernetes.io/projected/341bef6c-9dd7-4f4c-b1cc-dc69e421ab98-kube-api-access-rtpqn\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:26:43.494994 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:43.494958    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb" path="/var/lib/kubelet/pods/1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb/volumes"
Sep 17 14:26:44.021816 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:44.021781    2582 generic.go:356] "Generic (PLEG): container finished" podID="341bef6c-9dd7-4f4c-b1cc-dc69e421ab98" containerID="3f94109a0eaa3e7b9131334f8ee30f322f1298a2465b86abe162f8a874e5182b" exitCode=0
Sep 17 14:26:44.022000 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:44.021826    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-0c1110/noten-model-0c1110-kserve-5797796699-d6ssx"
Sep 17 14:26:44.022000 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:44.021869    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-0c1110/noten-model-0c1110-kserve-5797796699-d6ssx" event={"ID":"341bef6c-9dd7-4f4c-b1cc-dc69e421ab98","Type":"ContainerDied","Data":"3f94109a0eaa3e7b9131334f8ee30f322f1298a2465b86abe162f8a874e5182b"}
Sep 17 14:26:44.022000 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:44.021914    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-0c1110/noten-model-0c1110-kserve-5797796699-d6ssx" event={"ID":"341bef6c-9dd7-4f4c-b1cc-dc69e421ab98","Type":"ContainerDied","Data":"b626dccfffd31f994641915c78cc7a363d2c336a19c3301aad4ece839682008f"}
Sep 17 14:26:44.022000 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:44.021930    2582 scope.go:117] "RemoveContainer" containerID="3f94109a0eaa3e7b9131334f8ee30f322f1298a2465b86abe162f8a874e5182b"
Sep 17 14:26:44.033413 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:44.033391    2582 scope.go:117] "RemoveContainer" containerID="3f94109a0eaa3e7b9131334f8ee30f322f1298a2465b86abe162f8a874e5182b"
Sep 17 14:26:44.033756 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:26:44.033735    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f94109a0eaa3e7b9131334f8ee30f322f1298a2465b86abe162f8a874e5182b\": container with ID starting with 3f94109a0eaa3e7b9131334f8ee30f322f1298a2465b86abe162f8a874e5182b not found: ID does not exist" containerID="3f94109a0eaa3e7b9131334f8ee30f322f1298a2465b86abe162f8a874e5182b"
Sep 17 14:26:44.033828 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:44.033765    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f94109a0eaa3e7b9131334f8ee30f322f1298a2465b86abe162f8a874e5182b"} err="failed to get container status \"3f94109a0eaa3e7b9131334f8ee30f322f1298a2465b86abe162f8a874e5182b\": rpc error: code = NotFound desc = could not find container \"3f94109a0eaa3e7b9131334f8ee30f322f1298a2465b86abe162f8a874e5182b\": container with ID starting with 3f94109a0eaa3e7b9131334f8ee30f322f1298a2465b86abe162f8a874e5182b not found: ID does not exist"
Sep 17 14:26:44.038278 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:44.038244    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-0c1110/noten-model-0c1110-kserve-5797796699-d6ssx"]
Sep 17 14:26:44.044341 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:44.044319    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-0c1110/noten-model-0c1110-kserve-5797796699-d6ssx"]
Sep 17 14:26:45.494040 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:45.494007    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="341bef6c-9dd7-4f4c-b1cc-dc69e421ab98" path="/var/lib/kubelet/pods/341bef6c-9dd7-4f4c-b1cc-dc69e421ab98/volumes"
Sep 17 14:26:47.354109 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:47.354074    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:26:47.524803 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:47.524767    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn"]
Sep 17 14:26:47.525034 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:47.525008    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn" podUID="725b0118-b60a-4972-a84a-d52af1642e0b" containerName="istio-proxy" containerID="cri-o://0af183ee7dd9bde16635b6bc482cde3e139067e5bc2acaa05ddfaf1e2a2afd24" gracePeriod=30
Sep 17 14:26:47.578910 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:47.578875    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:26:47.875597 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:47.875561    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b8115-wkhhg"]
Sep 17 14:26:47.876046 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:47.876032    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb" containerName="main"
Sep 17 14:26:47.876088 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:47.876048    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb" containerName="main"
Sep 17 14:26:47.876088 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:47.876062    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="341bef6c-9dd7-4f4c-b1cc-dc69e421ab98" containerName="main"
Sep 17 14:26:47.876088 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:47.876067    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="341bef6c-9dd7-4f4c-b1cc-dc69e421ab98" containerName="main"
Sep 17 14:26:47.876181 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:47.876136    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="341bef6c-9dd7-4f4c-b1cc-dc69e421ab98" containerName="main"
Sep 17 14:26:47.876181 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:47.876148    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d683c3f-6bfc-44a5-aa45-d9956a9ee5fb" containerName="main"
Sep 17 14:26:47.878831 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:47.878813    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b8115-wkhhg"
Sep 17 14:26:47.881279 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:47.881256    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ddwrn\""
Sep 17 14:26:47.887883 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:47.887858    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b8115-wkhhg"]
Sep 17 14:26:47.984366 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:47.984329    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9994fea7-15b5-4871-91e9-a41fa86eef10-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-9b8115-wkhhg\" (UID: \"9994fea7-15b5-4871-91e9-a41fa86eef10\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b8115-wkhhg"
Sep 17 14:26:48.085134 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:48.085092    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9994fea7-15b5-4871-91e9-a41fa86eef10-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-9b8115-wkhhg\" (UID: \"9994fea7-15b5-4871-91e9-a41fa86eef10\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b8115-wkhhg"
Sep 17 14:26:48.085785 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:48.085761    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9994fea7-15b5-4871-91e9-a41fa86eef10-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-9b8115-wkhhg\" (UID: \"9994fea7-15b5-4871-91e9-a41fa86eef10\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b8115-wkhhg"
Sep 17 14:26:48.190756 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:48.190670    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b8115-wkhhg"
Sep 17 14:26:48.194756 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:48.194729    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-1a095586/e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7"
Sep 17 14:26:48.194878 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:48.194772    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-1a095586/e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7"
Sep 17 14:26:48.209671 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:48.209646    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-1a095586/e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7"
Sep 17 14:26:48.213703 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:48.213678    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:26:48.283090 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:48.283058    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-b-9497140e-kserve-794df4446b-stw2n"]
Sep 17 14:26:48.287105 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:48.287080    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-b-9497140e-kserve-794df4446b-stw2n"
Sep 17 14:26:48.289808 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:48.289786    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-fcd9f3\"/\"e2e-conflict-b-9497140e-kserve-self-signed-certs\""
Sep 17 14:26:48.299967 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:48.299916    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-b-9497140e-kserve-794df4446b-stw2n"]
Sep 17 14:26:48.357305 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:48.357257    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:26:48.388508 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:48.388474    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b259eb30-d5e7-4fde-b353-9dbcac62ece9-model-cache\") pod \"e2e-conflict-b-9497140e-kserve-794df4446b-stw2n\" (UID: \"b259eb30-d5e7-4fde-b353-9dbcac62ece9\") " pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-b-9497140e-kserve-794df4446b-stw2n"
Sep 17 14:26:48.388717 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:48.388535    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b259eb30-d5e7-4fde-b353-9dbcac62ece9-tls-certs\") pod \"e2e-conflict-b-9497140e-kserve-794df4446b-stw2n\" (UID: \"b259eb30-d5e7-4fde-b353-9dbcac62ece9\") " pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-b-9497140e-kserve-794df4446b-stw2n"
Sep 17 14:26:48.388717 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:48.388580    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b259eb30-d5e7-4fde-b353-9dbcac62ece9-tmp-dir\") pod \"e2e-conflict-b-9497140e-kserve-794df4446b-stw2n\" (UID: \"b259eb30-d5e7-4fde-b353-9dbcac62ece9\") " pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-b-9497140e-kserve-794df4446b-stw2n"
Sep 17 14:26:48.388717 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:48.388632    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b259eb30-d5e7-4fde-b353-9dbcac62ece9-home\") pod \"e2e-conflict-b-9497140e-kserve-794df4446b-stw2n\" (UID: \"b259eb30-d5e7-4fde-b353-9dbcac62ece9\") " pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-b-9497140e-kserve-794df4446b-stw2n"
Sep 17 14:26:48.388717 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:48.388674    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jbww\" (UniqueName: \"kubernetes.io/projected/b259eb30-d5e7-4fde-b353-9dbcac62ece9-kube-api-access-9jbww\") pod \"e2e-conflict-b-9497140e-kserve-794df4446b-stw2n\" (UID: \"b259eb30-d5e7-4fde-b353-9dbcac62ece9\") " pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-b-9497140e-kserve-794df4446b-stw2n"
Sep 17 14:26:48.388717 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:48.388691    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b259eb30-d5e7-4fde-b353-9dbcac62ece9-dshm\") pod \"e2e-conflict-b-9497140e-kserve-794df4446b-stw2n\" (UID: \"b259eb30-d5e7-4fde-b353-9dbcac62ece9\") " pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-b-9497140e-kserve-794df4446b-stw2n"
Sep 17 14:26:48.486035 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:26:48.485999    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9994fea7_15b5_4871_91e9_a41fa86eef10.slice/crio-baa4f26a24f5702abc32ee3c38d0c2f4f10559e591543dca9dac744924e5a478 WatchSource:0}: Error finding container baa4f26a24f5702abc32ee3c38d0c2f4f10559e591543dca9dac744924e5a478: Status 404 returned error can't find the container with id baa4f26a24f5702abc32ee3c38d0c2f4f10559e591543dca9dac744924e5a478
Sep 17 14:26:48.486310 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:48.486264    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b8115-wkhhg"]
Sep 17 14:26:48.490255 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:48.489305    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b259eb30-d5e7-4fde-b353-9dbcac62ece9-tls-certs\") pod \"e2e-conflict-b-9497140e-kserve-794df4446b-stw2n\" (UID: \"b259eb30-d5e7-4fde-b353-9dbcac62ece9\") " pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-b-9497140e-kserve-794df4446b-stw2n"
Sep 17 14:26:48.490255 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:48.489350    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b259eb30-d5e7-4fde-b353-9dbcac62ece9-tmp-dir\") pod \"e2e-conflict-b-9497140e-kserve-794df4446b-stw2n\" (UID: \"b259eb30-d5e7-4fde-b353-9dbcac62ece9\") " pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-b-9497140e-kserve-794df4446b-stw2n"
Sep 17 14:26:48.490255 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:48.489389    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b259eb30-d5e7-4fde-b353-9dbcac62ece9-home\") pod \"e2e-conflict-b-9497140e-kserve-794df4446b-stw2n\" (UID: \"b259eb30-d5e7-4fde-b353-9dbcac62ece9\") " pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-b-9497140e-kserve-794df4446b-stw2n"
Sep 17 14:26:48.490255 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:48.489445    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jbww\" (UniqueName: \"kubernetes.io/projected/b259eb30-d5e7-4fde-b353-9dbcac62ece9-kube-api-access-9jbww\") pod \"e2e-conflict-b-9497140e-kserve-794df4446b-stw2n\" (UID: \"b259eb30-d5e7-4fde-b353-9dbcac62ece9\") " pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-b-9497140e-kserve-794df4446b-stw2n"
Sep 17 14:26:48.490255 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:48.489471    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b259eb30-d5e7-4fde-b353-9dbcac62ece9-dshm\") pod \"e2e-conflict-b-9497140e-kserve-794df4446b-stw2n\" (UID: \"b259eb30-d5e7-4fde-b353-9dbcac62ece9\") " pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-b-9497140e-kserve-794df4446b-stw2n"
Sep 17 14:26:48.490255 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:48.489538    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b259eb30-d5e7-4fde-b353-9dbcac62ece9-model-cache\") pod \"e2e-conflict-b-9497140e-kserve-794df4446b-stw2n\" (UID: \"b259eb30-d5e7-4fde-b353-9dbcac62ece9\") " pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-b-9497140e-kserve-794df4446b-stw2n"
Sep 17 14:26:48.490255 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:48.489989    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b259eb30-d5e7-4fde-b353-9dbcac62ece9-model-cache\") pod \"e2e-conflict-b-9497140e-kserve-794df4446b-stw2n\" (UID: \"b259eb30-d5e7-4fde-b353-9dbcac62ece9\") " pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-b-9497140e-kserve-794df4446b-stw2n"
Sep 17 14:26:48.490255 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:48.490065    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b259eb30-d5e7-4fde-b353-9dbcac62ece9-home\") pod \"e2e-conflict-b-9497140e-kserve-794df4446b-stw2n\" (UID: \"b259eb30-d5e7-4fde-b353-9dbcac62ece9\") " pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-b-9497140e-kserve-794df4446b-stw2n"
Sep 17 14:26:48.490824 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:48.490310    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b259eb30-d5e7-4fde-b353-9dbcac62ece9-tmp-dir\") pod \"e2e-conflict-b-9497140e-kserve-794df4446b-stw2n\" (UID: \"b259eb30-d5e7-4fde-b353-9dbcac62ece9\") " pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-b-9497140e-kserve-794df4446b-stw2n"
Sep 17 14:26:48.494009 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:48.493499    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b259eb30-d5e7-4fde-b353-9dbcac62ece9-dshm\") pod \"e2e-conflict-b-9497140e-kserve-794df4446b-stw2n\" (UID: \"b259eb30-d5e7-4fde-b353-9dbcac62ece9\") " pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-b-9497140e-kserve-794df4446b-stw2n"
Sep 17 14:26:48.494561 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:48.494518    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b259eb30-d5e7-4fde-b353-9dbcac62ece9-tls-certs\") pod \"e2e-conflict-b-9497140e-kserve-794df4446b-stw2n\" (UID: \"b259eb30-d5e7-4fde-b353-9dbcac62ece9\") " pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-b-9497140e-kserve-794df4446b-stw2n"
Sep 17 14:26:48.499998 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:48.499973    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jbww\" (UniqueName: \"kubernetes.io/projected/b259eb30-d5e7-4fde-b353-9dbcac62ece9-kube-api-access-9jbww\") pod \"e2e-conflict-b-9497140e-kserve-794df4446b-stw2n\" (UID: \"b259eb30-d5e7-4fde-b353-9dbcac62ece9\") " pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-b-9497140e-kserve-794df4446b-stw2n"
Sep 17 14:26:48.600754 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:48.600711    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-b-9497140e-kserve-794df4446b-stw2n"
Sep 17 14:26:48.878563 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:48.878537    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-b-9497140e-kserve-794df4446b-stw2n"]
Sep 17 14:26:48.879846 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:26:48.879812    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb259eb30_d5e7_4fde_b353_9dbcac62ece9.slice/crio-d2f65379ba3603eb726134e2a8462b533508e5c7f0cadb45ffcdd914b349f8b8 WatchSource:0}: Error finding container d2f65379ba3603eb726134e2a8462b533508e5c7f0cadb45ffcdd914b349f8b8: Status 404 returned error can't find the container with id d2f65379ba3603eb726134e2a8462b533508e5c7f0cadb45ffcdd914b349f8b8
Sep 17 14:26:49.055735 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:49.055647    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-b-9497140e-kserve-794df4446b-stw2n" event={"ID":"b259eb30-d5e7-4fde-b353-9dbcac62ece9","Type":"ContainerStarted","Data":"f0012f882a2fe4cd8f77b4d57cf8d6e18da465f5197f638964cc716bcf363bfb"}
Sep 17 14:26:49.055735 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:49.055692    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-b-9497140e-kserve-794df4446b-stw2n" event={"ID":"b259eb30-d5e7-4fde-b353-9dbcac62ece9","Type":"ContainerStarted","Data":"d2f65379ba3603eb726134e2a8462b533508e5c7f0cadb45ffcdd914b349f8b8"}
Sep 17 14:26:49.057308 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:49.057280    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b8115-wkhhg" event={"ID":"9994fea7-15b5-4871-91e9-a41fa86eef10","Type":"ContainerStarted","Data":"4cbc6fd62788bf54bd8551df69525166e8b98aea0d7aaa558fee592a3752c3af"}
Sep 17 14:26:49.057440 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:49.057352    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b8115-wkhhg" event={"ID":"9994fea7-15b5-4871-91e9-a41fa86eef10","Type":"ContainerStarted","Data":"baa4f26a24f5702abc32ee3c38d0c2f4f10559e591543dca9dac744924e5a478"}
Sep 17 14:26:49.068849 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:49.068818    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-1a095586/e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7"
Sep 17 14:26:49.075272 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:49.075231    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-b-9497140e-kserve-794df4446b-stw2n" podStartSLOduration=1.075217084 podStartE2EDuration="1.075217084s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:26:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:26:49.074211494 +0000 UTC m=+1310.075754969" watchObservedRunningTime="2026-09-17 14:26:49.075217084 +0000 UTC m=+1310.076760558"
Sep 17 14:26:49.113710 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:49.113659    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b8115-wkhhg" podStartSLOduration=2.113642554 podStartE2EDuration="2.113642554s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:26:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:26:49.112082274 +0000 UTC m=+1310.113625748" watchObservedRunningTime="2026-09-17 14:26:49.113642554 +0000 UTC m=+1310.115186027"
Sep 17 14:26:49.144260 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:49.144196    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:26:49.818098 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:49.818052    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:26:50.066052 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:50.066016    2582 generic.go:356] "Generic (PLEG): container finished" podID="9994fea7-15b5-4871-91e9-a41fa86eef10" containerID="4cbc6fd62788bf54bd8551df69525166e8b98aea0d7aaa558fee592a3752c3af" exitCode=0
Sep 17 14:26:50.066265 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:50.066097    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b8115-wkhhg" event={"ID":"9994fea7-15b5-4871-91e9-a41fa86eef10","Type":"ContainerDied","Data":"4cbc6fd62788bf54bd8551df69525166e8b98aea0d7aaa558fee592a3752c3af"}
Sep 17 14:26:50.681909 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:50.681836    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:26:51.228919 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:51.227377    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b8115-wkhhg"
Sep 17 14:26:51.322880 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:51.322840    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9994fea7-15b5-4871-91e9-a41fa86eef10-maas-api-service-ca\") pod \"9994fea7-15b5-4871-91e9-a41fa86eef10\" (UID: \"9994fea7-15b5-4871-91e9-a41fa86eef10\") "
Sep 17 14:26:51.323150 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:51.323129    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9994fea7-15b5-4871-91e9-a41fa86eef10-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9994fea7-15b5-4871-91e9-a41fa86eef10" (UID: "9994fea7-15b5-4871-91e9-a41fa86eef10"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:26:51.323347 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:51.323322    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9994fea7-15b5-4871-91e9-a41fa86eef10-maas-api-service-ca\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:26:51.412925 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:51.412791    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:26:52.082455 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:52.082422    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b8115-wkhhg" event={"ID":"9994fea7-15b5-4871-91e9-a41fa86eef10","Type":"ContainerDied","Data":"baa4f26a24f5702abc32ee3c38d0c2f4f10559e591543dca9dac744924e5a478"}
Sep 17 14:26:52.082455 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:52.082448    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b8115-wkhhg"
Sep 17 14:26:52.082697 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:52.082479    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="baa4f26a24f5702abc32ee3c38d0c2f4f10559e591543dca9dac744924e5a478"
Sep 17 14:26:52.126437 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:52.126403    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:26:53.046974 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:53.046941    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:26:53.090602 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:53.090569    2582 generic.go:356] "Generic (PLEG): container finished" podID="725b0118-b60a-4972-a84a-d52af1642e0b" containerID="0af183ee7dd9bde16635b6bc482cde3e139067e5bc2acaa05ddfaf1e2a2afd24" exitCode=0
Sep 17 14:26:53.090789 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:53.090640    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn" event={"ID":"725b0118-b60a-4972-a84a-d52af1642e0b","Type":"ContainerDied","Data":"0af183ee7dd9bde16635b6bc482cde3e139067e5bc2acaa05ddfaf1e2a2afd24"}
Sep 17 14:26:53.090789 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:53.090703    2582 scope.go:117] "RemoveContainer" containerID="a7dfc2b887f2083f84aa4641efa503bcf0a4325a9468bdfd34fd53fda29ef71b"
Sep 17 14:26:53.196955 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:53.196933    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn"
Sep 17 14:26:53.245920 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:53.245884    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8n5ww\" (UniqueName: \"kubernetes.io/projected/725b0118-b60a-4972-a84a-d52af1642e0b-kube-api-access-8n5ww\") pod \"725b0118-b60a-4972-a84a-d52af1642e0b\" (UID: \"725b0118-b60a-4972-a84a-d52af1642e0b\") "
Sep 17 14:26:53.246123 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:53.245932    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/725b0118-b60a-4972-a84a-d52af1642e0b-credential-socket\") pod \"725b0118-b60a-4972-a84a-d52af1642e0b\" (UID: \"725b0118-b60a-4972-a84a-d52af1642e0b\") "
Sep 17 14:26:53.246123 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:53.245965    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/725b0118-b60a-4972-a84a-d52af1642e0b-istio-data\") pod \"725b0118-b60a-4972-a84a-d52af1642e0b\" (UID: \"725b0118-b60a-4972-a84a-d52af1642e0b\") "
Sep 17 14:26:53.246123 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:53.246017    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/725b0118-b60a-4972-a84a-d52af1642e0b-istio-token\") pod \"725b0118-b60a-4972-a84a-d52af1642e0b\" (UID: \"725b0118-b60a-4972-a84a-d52af1642e0b\") "
Sep 17 14:26:53.246123 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:53.246069    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/725b0118-b60a-4972-a84a-d52af1642e0b-istiod-ca-cert\") pod \"725b0118-b60a-4972-a84a-d52af1642e0b\" (UID: \"725b0118-b60a-4972-a84a-d52af1642e0b\") "
Sep 17 14:26:53.246123 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:53.246122    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/725b0118-b60a-4972-a84a-d52af1642e0b-workload-certs\") pod \"725b0118-b60a-4972-a84a-d52af1642e0b\" (UID: \"725b0118-b60a-4972-a84a-d52af1642e0b\") "
Sep 17 14:26:53.246426 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:53.246167    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/725b0118-b60a-4972-a84a-d52af1642e0b-istio-podinfo\") pod \"725b0118-b60a-4972-a84a-d52af1642e0b\" (UID: \"725b0118-b60a-4972-a84a-d52af1642e0b\") "
Sep 17 14:26:53.246426 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:53.246252    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/725b0118-b60a-4972-a84a-d52af1642e0b-workload-socket\") pod \"725b0118-b60a-4972-a84a-d52af1642e0b\" (UID: \"725b0118-b60a-4972-a84a-d52af1642e0b\") "
Sep 17 14:26:53.246426 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:53.246319    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/725b0118-b60a-4972-a84a-d52af1642e0b-istio-envoy\") pod \"725b0118-b60a-4972-a84a-d52af1642e0b\" (UID: \"725b0118-b60a-4972-a84a-d52af1642e0b\") "
Sep 17 14:26:53.246586 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:53.246245    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/725b0118-b60a-4972-a84a-d52af1642e0b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "725b0118-b60a-4972-a84a-d52af1642e0b" (UID: "725b0118-b60a-4972-a84a-d52af1642e0b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:26:53.246586 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:53.246422    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/725b0118-b60a-4972-a84a-d52af1642e0b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "725b0118-b60a-4972-a84a-d52af1642e0b" (UID: "725b0118-b60a-4972-a84a-d52af1642e0b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:26:53.246586 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:53.246545    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/725b0118-b60a-4972-a84a-d52af1642e0b-istio-data" (OuterVolumeSpecName: "istio-data") pod "725b0118-b60a-4972-a84a-d52af1642e0b" (UID: "725b0118-b60a-4972-a84a-d52af1642e0b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:26:53.246798 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:53.246744    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/725b0118-b60a-4972-a84a-d52af1642e0b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "725b0118-b60a-4972-a84a-d52af1642e0b" (UID: "725b0118-b60a-4972-a84a-d52af1642e0b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:26:53.246798 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:53.246763    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/725b0118-b60a-4972-a84a-d52af1642e0b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "725b0118-b60a-4972-a84a-d52af1642e0b" (UID: "725b0118-b60a-4972-a84a-d52af1642e0b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:26:53.246909 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:53.246812    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/725b0118-b60a-4972-a84a-d52af1642e0b-credential-socket\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:26:53.246909 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:53.246833    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/725b0118-b60a-4972-a84a-d52af1642e0b-istio-data\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:26:53.246909 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:53.246848    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/725b0118-b60a-4972-a84a-d52af1642e0b-workload-certs\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:26:53.248795 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:53.248757    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/725b0118-b60a-4972-a84a-d52af1642e0b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "725b0118-b60a-4972-a84a-d52af1642e0b" (UID: "725b0118-b60a-4972-a84a-d52af1642e0b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:26:53.248795 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:53.248777    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/725b0118-b60a-4972-a84a-d52af1642e0b-istio-token" (OuterVolumeSpecName: "istio-token") pod "725b0118-b60a-4972-a84a-d52af1642e0b" (UID: "725b0118-b60a-4972-a84a-d52af1642e0b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:26:53.248955 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:53.248899    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/725b0118-b60a-4972-a84a-d52af1642e0b-kube-api-access-8n5ww" (OuterVolumeSpecName: "kube-api-access-8n5ww") pod "725b0118-b60a-4972-a84a-d52af1642e0b" (UID: "725b0118-b60a-4972-a84a-d52af1642e0b"). InnerVolumeSpecName "kube-api-access-8n5ww". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:26:53.249270 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:53.249250    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/725b0118-b60a-4972-a84a-d52af1642e0b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "725b0118-b60a-4972-a84a-d52af1642e0b" (UID: "725b0118-b60a-4972-a84a-d52af1642e0b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 14:26:53.348205 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:53.348125    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/725b0118-b60a-4972-a84a-d52af1642e0b-istio-podinfo\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:26:53.348205 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:53.348151    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/725b0118-b60a-4972-a84a-d52af1642e0b-workload-socket\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:26:53.348205 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:53.348161    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/725b0118-b60a-4972-a84a-d52af1642e0b-istio-envoy\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:26:53.348205 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:53.348170    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8n5ww\" (UniqueName: \"kubernetes.io/projected/725b0118-b60a-4972-a84a-d52af1642e0b-kube-api-access-8n5ww\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:26:53.348205 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:53.348180    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/725b0118-b60a-4972-a84a-d52af1642e0b-istio-token\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:26:53.348205 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:53.348190    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/725b0118-b60a-4972-a84a-d52af1642e0b-istiod-ca-cert\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:26:53.709972 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:53.709935    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:26:54.098070 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:54.097980    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn"
Sep 17 14:26:54.098070 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:54.098005    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn" event={"ID":"725b0118-b60a-4972-a84a-d52af1642e0b","Type":"ContainerDied","Data":"685840ed8f4e1929e668466be4a5b823c17a572ce9c4eb2e37312aed153d0cc0"}
Sep 17 14:26:54.098579 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:54.098071    2582 scope.go:117] "RemoveContainer" containerID="0af183ee7dd9bde16635b6bc482cde3e139067e5bc2acaa05ddfaf1e2a2afd24"
Sep 17 14:26:54.116497 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:54.116472    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn"]
Sep 17 14:26:54.118933 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:54.118912    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-181b16-openshift-default-d46d6599b-2fkkn"]
Sep 17 14:26:55.494370 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:55.494339    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="725b0118-b60a-4972-a84a-d52af1642e0b" path="/var/lib/kubelet/pods/725b0118-b60a-4972-a84a-d52af1642e0b/volumes"
Sep 17 14:26:58.557270 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:58.557234    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:26:58.601255 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:58.601220    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-b-9497140e-kserve-794df4446b-stw2n"
Sep 17 14:26:58.601503 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:58.601487    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-b-9497140e-kserve-794df4446b-stw2n"
Sep 17 14:26:58.617426 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:58.617395    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-b-9497140e-kserve-794df4446b-stw2n"
Sep 17 14:26:58.656516 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:58.656488    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:26:58.756741 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:58.756707    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:26:58.774620 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:58.774588    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b8115-wkhhg"]
Sep 17 14:26:58.777965 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:58.777942    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b8115-wkhhg"]
Sep 17 14:26:58.858746 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:58.858651    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:26:59.143517 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:59.143429    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-b-9497140e-kserve-794df4446b-stw2n"
Sep 17 14:26:59.198577 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:59.198528    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:26:59.309687 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:59.309655    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:26:59.495144 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:59.495107    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9994fea7-15b5-4871-91e9-a41fa86eef10" path="/var/lib/kubelet/pods/9994fea7-15b5-4871-91e9-a41fa86eef10/volumes"
Sep 17 14:26:59.610217 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:59.610183    2582 scope.go:117] "RemoveContainer" containerID="06f8c189b5755f666b171414759c1def338f76c148f209721d4f676dc2ec2e15"
Sep 17 14:26:59.619521 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:59.619500    2582 scope.go:117] "RemoveContainer" containerID="5d5d6f9f13da2ae499c4fffd447179e123f45d189ae73cb98f426b9111d2acd3"
Sep 17 14:26:59.627473 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:26:59.627452    2582 scope.go:117] "RemoveContainer" containerID="42541548cda2927085935dae6380265c0cbc41cf965e1123bb4d107414e29388"
Sep 17 14:27:00.057039 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:00.057000    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:00.915220 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:00.915186    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:01.659169 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:01.659117    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:02.360190 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:02.360146    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:03.047913 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:03.047872    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:05.253790 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:05.253755    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:05.358514 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:05.358475    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:05.545850 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:05.545770    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:06.214633 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.214585    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:06.359515 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.359456    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr"]
Sep 17 14:27:06.360281 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.360257    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="725b0118-b60a-4972-a84a-d52af1642e0b" containerName="istio-proxy"
Sep 17 14:27:06.360379 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.360284    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="725b0118-b60a-4972-a84a-d52af1642e0b" containerName="istio-proxy"
Sep 17 14:27:06.360379 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.360299    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9994fea7-15b5-4871-91e9-a41fa86eef10" containerName="revoke-keys"
Sep 17 14:27:06.360379 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.360306    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9994fea7-15b5-4871-91e9-a41fa86eef10" containerName="revoke-keys"
Sep 17 14:27:06.360379 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.360319    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="725b0118-b60a-4972-a84a-d52af1642e0b" containerName="istio-proxy"
Sep 17 14:27:06.360379 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.360327    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="725b0118-b60a-4972-a84a-d52af1642e0b" containerName="istio-proxy"
Sep 17 14:27:06.360657 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.360471    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="725b0118-b60a-4972-a84a-d52af1642e0b" containerName="istio-proxy"
Sep 17 14:27:06.360657 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.360486    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9994fea7-15b5-4871-91e9-a41fa86eef10" containerName="revoke-keys"
Sep 17 14:27:06.363859 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.363838    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr"
Sep 17 14:27:06.366084 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.366063    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-cc3801-openshift-default-dockercfg-fc486\""
Sep 17 14:27:06.376062 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.376039    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr"]
Sep 17 14:27:06.445145 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.445113    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:06.487689 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.482341    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-istio-token\") pod \"e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr\" (UID: \"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff\") " pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr"
Sep 17 14:27:06.487689 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.482412    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-workload-certs\") pod \"e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr\" (UID: \"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff\") " pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr"
Sep 17 14:27:06.487689 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.482442    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-istiod-ca-cert\") pod \"e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr\" (UID: \"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff\") " pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr"
Sep 17 14:27:06.487689 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.482472    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-credential-socket\") pod \"e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr\" (UID: \"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff\") " pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr"
Sep 17 14:27:06.487689 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.482526    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-workload-socket\") pod \"e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr\" (UID: \"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff\") " pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr"
Sep 17 14:27:06.487689 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.482575    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-istio-podinfo\") pod \"e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr\" (UID: \"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff\") " pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr"
Sep 17 14:27:06.487689 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.482600    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-istio-envoy\") pod \"e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr\" (UID: \"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff\") " pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr"
Sep 17 14:27:06.487689 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.482692    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-istio-data\") pod \"e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr\" (UID: \"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff\") " pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr"
Sep 17 14:27:06.487689 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.482766    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwxt9\" (UniqueName: \"kubernetes.io/projected/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-kube-api-access-wwxt9\") pod \"e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr\" (UID: \"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff\") " pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr"
Sep 17 14:27:06.547004 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.546966    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:06.584037 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.584006    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-workload-socket\") pod \"e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr\" (UID: \"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff\") " pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr"
Sep 17 14:27:06.584173 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.584071    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-istio-podinfo\") pod \"e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr\" (UID: \"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff\") " pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr"
Sep 17 14:27:06.584173 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.584105    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-istio-envoy\") pod \"e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr\" (UID: \"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff\") " pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr"
Sep 17 14:27:06.584279 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.584178    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-istio-data\") pod \"e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr\" (UID: \"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff\") " pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr"
Sep 17 14:27:06.584279 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.584261    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwxt9\" (UniqueName: \"kubernetes.io/projected/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-kube-api-access-wwxt9\") pod \"e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr\" (UID: \"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff\") " pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr"
Sep 17 14:27:06.584385 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.584314    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-istio-token\") pod \"e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr\" (UID: \"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff\") " pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr"
Sep 17 14:27:06.584441 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.584400    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-workload-certs\") pod \"e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr\" (UID: \"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff\") " pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr"
Sep 17 14:27:06.584441 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.584427    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-istiod-ca-cert\") pod \"e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr\" (UID: \"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff\") " pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr"
Sep 17 14:27:06.584551 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.584457    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-credential-socket\") pod \"e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr\" (UID: \"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff\") " pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr"
Sep 17 14:27:06.584551 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.584458    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-workload-socket\") pod \"e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr\" (UID: \"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff\") " pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr"
Sep 17 14:27:06.584881 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.584825    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-workload-certs\") pod \"e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr\" (UID: \"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff\") " pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr"
Sep 17 14:27:06.585063 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.585037    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-istio-data\") pod \"e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr\" (UID: \"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff\") " pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr"
Sep 17 14:27:06.585155 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.585049    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-credential-socket\") pod \"e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr\" (UID: \"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff\") " pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr"
Sep 17 14:27:06.585262 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.585243    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-istiod-ca-cert\") pod \"e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr\" (UID: \"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff\") " pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr"
Sep 17 14:27:06.586558 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.586534    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-istio-envoy\") pod \"e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr\" (UID: \"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff\") " pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr"
Sep 17 14:27:06.586782 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.586767    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-istio-podinfo\") pod \"e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr\" (UID: \"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff\") " pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr"
Sep 17 14:27:06.591525 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.591504    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-istio-token\") pod \"e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr\" (UID: \"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff\") " pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr"
Sep 17 14:27:06.591901 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.591884    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwxt9\" (UniqueName: \"kubernetes.io/projected/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-kube-api-access-wwxt9\") pod \"e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr\" (UID: \"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff\") " pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr"
Sep 17 14:27:06.676551 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.676516    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr"
Sep 17 14:27:06.924437 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.924353    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:06.954299 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.954274    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr"]
Sep 17 14:27:06.956353 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:27:06.956319    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d7bc84f_6345_4a14_a73b_21e0aa0a30ff.slice/crio-97a109c91dd86c242577443266931fb6845ec87cf349473c1eb4686462c6ecf0 WatchSource:0}: Error finding container 97a109c91dd86c242577443266931fb6845ec87cf349473c1eb4686462c6ecf0: Status 404 returned error can't find the container with id 97a109c91dd86c242577443266931fb6845ec87cf349473c1eb4686462c6ecf0
Sep 17 14:27:06.959096 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.959064    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:27:06.959178 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.959130    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:27:06.959225 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:06.959176    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:27:07.180725 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:07.180623    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr" event={"ID":"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff","Type":"ContainerStarted","Data":"a4a20a5a971d62d9f1307dbdfd286435147255ae51792d1c0388e3ca5efbfd12"}
Sep 17 14:27:07.180725 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:07.180695    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr" event={"ID":"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff","Type":"ContainerStarted","Data":"97a109c91dd86c242577443266931fb6845ec87cf349473c1eb4686462c6ecf0"}
Sep 17 14:27:07.204768 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:07.204707    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr" podStartSLOduration=1.204686621 podStartE2EDuration="1.204686621s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:27:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:27:07.202751785 +0000 UTC m=+1328.204295260" watchObservedRunningTime="2026-09-17 14:27:07.204686621 +0000 UTC m=+1328.206230097"
Sep 17 14:27:07.677175 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:07.677143    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr"
Sep 17 14:27:08.530131 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:08.530083    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:08.683233 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:08.683200    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr"
Sep 17 14:27:09.160043 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:09.160003    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:09.193350 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:09.193321    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr"
Sep 17 14:27:09.194268 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:09.194248    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr"
Sep 17 14:27:09.259693 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:09.259660    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:09.361333 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:09.361284    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:10.244544 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:10.244508    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:11.054557 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:11.054524    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:11.867361 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:11.867323    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:12.619269 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:12.619229    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:13.446056 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:13.446021    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:13.548202 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:13.548170    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:13.977761 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:13.977726    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:14.070849 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:14.070810    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:14.253251 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:14.253168    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:27.653348 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:27.653246    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a095586-6k7cv"]
Sep 17 14:27:27.654161 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:27.654131    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="725b0118-b60a-4972-a84a-d52af1642e0b" containerName="istio-proxy"
Sep 17 14:27:27.657955 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:27.657931    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a095586-6k7cv"
Sep 17 14:27:27.660751 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:27.660731    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ddwrn\""
Sep 17 14:27:27.667124 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:27.667098    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a095586-6k7cv"]
Sep 17 14:27:27.716308 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:27.716256    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52beca38-a48f-4292-b94a-92671dc121bc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-1a095586-6k7cv\" (UID: \"52beca38-a48f-4292-b94a-92671dc121bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a095586-6k7cv"
Sep 17 14:27:27.817727 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:27.817693    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52beca38-a48f-4292-b94a-92671dc121bc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-1a095586-6k7cv\" (UID: \"52beca38-a48f-4292-b94a-92671dc121bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a095586-6k7cv"
Sep 17 14:27:27.818319 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:27.818301    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52beca38-a48f-4292-b94a-92671dc121bc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-1a095586-6k7cv\" (UID: \"52beca38-a48f-4292-b94a-92671dc121bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a095586-6k7cv"
Sep 17 14:27:27.834922 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:27.834884    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:27.970726 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:27.970692    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a095586-6k7cv"
Sep 17 14:27:28.254795 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:28.254704    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a095586-6k7cv"]
Sep 17 14:27:28.259840 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:27:28.259805    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52beca38_a48f_4292_b94a_92671dc121bc.slice/crio-1768d610eb34eb27f9b692b94f7dce70b4f2c677b59c94a644c3e42eb57d0e8f WatchSource:0}: Error finding container 1768d610eb34eb27f9b692b94f7dce70b4f2c677b59c94a644c3e42eb57d0e8f: Status 404 returned error can't find the container with id 1768d610eb34eb27f9b692b94f7dce70b4f2c677b59c94a644c3e42eb57d0e8f
Sep 17 14:27:28.313041 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:28.313016    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a095586-6k7cv" event={"ID":"52beca38-a48f-4292-b94a-92671dc121bc","Type":"ContainerStarted","Data":"1768d610eb34eb27f9b692b94f7dce70b4f2c677b59c94a644c3e42eb57d0e8f"}
Sep 17 14:27:28.662272 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:28.662241    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:29.323641 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:29.323591    2582 generic.go:356] "Generic (PLEG): container finished" podID="52beca38-a48f-4292-b94a-92671dc121bc" containerID="a09de9490b39a99a7931a003210011b3f99e186882d8249c8c4e4128118c2295" exitCode=0
Sep 17 14:27:29.323811 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:29.323679    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a095586-6k7cv" event={"ID":"52beca38-a48f-4292-b94a-92671dc121bc","Type":"ContainerDied","Data":"a09de9490b39a99a7931a003210011b3f99e186882d8249c8c4e4128118c2295"}
Sep 17 14:27:29.474728 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:29.474684    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:30.255048 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:30.255012    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:30.496425 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:30.496394    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a095586-6k7cv"
Sep 17 14:27:30.549905 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:30.549820    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52beca38-a48f-4292-b94a-92671dc121bc-maas-api-service-ca\") pod \"52beca38-a48f-4292-b94a-92671dc121bc\" (UID: \"52beca38-a48f-4292-b94a-92671dc121bc\") "
Sep 17 14:27:30.550530 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:30.550503    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52beca38-a48f-4292-b94a-92671dc121bc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "52beca38-a48f-4292-b94a-92671dc121bc" (UID: "52beca38-a48f-4292-b94a-92671dc121bc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:27:30.651402 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:30.651352    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52beca38-a48f-4292-b94a-92671dc121bc-maas-api-service-ca\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:27:30.926867 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:30.926774    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:31.341235 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:31.341194    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a095586-6k7cv" event={"ID":"52beca38-a48f-4292-b94a-92671dc121bc","Type":"ContainerDied","Data":"1768d610eb34eb27f9b692b94f7dce70b4f2c677b59c94a644c3e42eb57d0e8f"}
Sep 17 14:27:31.341594 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:31.341214    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a095586-6k7cv"
Sep 17 14:27:31.341594 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:31.341295    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1768d610eb34eb27f9b692b94f7dce70b4f2c677b59c94a644c3e42eb57d0e8f"
Sep 17 14:27:31.708990 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:31.708955    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:32.456764 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:32.456705    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:33.111207 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:33.111170    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:33.356086 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:33.356005    2582 generic.go:356] "Generic (PLEG): container finished" podID="7d7bc84f-6345-4a14-a73b-21e0aa0a30ff" containerID="a4a20a5a971d62d9f1307dbdfd286435147255ae51792d1c0388e3ca5efbfd12" exitCode=0
Sep 17 14:27:33.356086 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:33.356057    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr" event={"ID":"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff","Type":"ContainerDied","Data":"a4a20a5a971d62d9f1307dbdfd286435147255ae51792d1c0388e3ca5efbfd12"}
Sep 17 14:27:33.356501 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:33.356485    2582 scope.go:117] "RemoveContainer" containerID="a4a20a5a971d62d9f1307dbdfd286435147255ae51792d1c0388e3ca5efbfd12"
Sep 17 14:27:33.357719 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:33.357688    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:27:33.357778 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:33.357762    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:27:33.357814 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:33.357800    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:27:33.936812 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:33.936767    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:34.147138 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:34.146998    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:34.364918 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:34.364886    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr" event={"ID":"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff","Type":"ContainerStarted","Data":"b1ce9c406d84163c001f8f324b4a6d479e408f829427cfc48b88efef00170171"}
Sep 17 14:27:34.451107 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:34.451072    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:34.676932 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:34.676839    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr"
Sep 17 14:27:34.678010 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:34.677981    2582 patch_prober.go:28] interesting pod/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Sep 17 14:27:34.678152 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:34.678044    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr" podUID="7d7bc84f-6345-4a14-a73b-21e0aa0a30ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Sep 17 14:27:34.724525 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:34.724492    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:34.751380 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:34.751338    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd"]
Sep 17 14:27:34.752346 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:34.752318    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52beca38-a48f-4292-b94a-92671dc121bc" containerName="revoke-keys"
Sep 17 14:27:34.752495 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:34.752482    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="52beca38-a48f-4292-b94a-92671dc121bc" containerName="revoke-keys"
Sep 17 14:27:34.754633 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:34.752762    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="52beca38-a48f-4292-b94a-92671dc121bc" containerName="revoke-keys"
Sep 17 14:27:34.761996 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:34.761962    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd"
Sep 17 14:27:34.765989 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:34.765957    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-4c29a2-openshift-default-dockercfg-b6528\""
Sep 17 14:27:34.775626 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:34.775577    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd"]
Sep 17 14:27:34.899408 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:34.898670    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d882585-6a65-4e36-b1c7-7ec4562cd182-workload-certs\") pod \"e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd\" (UID: \"9d882585-6a65-4e36-b1c7-7ec4562cd182\") " pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd"
Sep 17 14:27:34.899408 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:34.898735    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d882585-6a65-4e36-b1c7-7ec4562cd182-istiod-ca-cert\") pod \"e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd\" (UID: \"9d882585-6a65-4e36-b1c7-7ec4562cd182\") " pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd"
Sep 17 14:27:34.899408 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:34.898895    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d882585-6a65-4e36-b1c7-7ec4562cd182-workload-socket\") pod \"e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd\" (UID: \"9d882585-6a65-4e36-b1c7-7ec4562cd182\") " pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd"
Sep 17 14:27:34.899408 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:34.898946    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d882585-6a65-4e36-b1c7-7ec4562cd182-istio-podinfo\") pod \"e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd\" (UID: \"9d882585-6a65-4e36-b1c7-7ec4562cd182\") " pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd"
Sep 17 14:27:34.899408 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:34.899036    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d882585-6a65-4e36-b1c7-7ec4562cd182-istio-token\") pod \"e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd\" (UID: \"9d882585-6a65-4e36-b1c7-7ec4562cd182\") " pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd"
Sep 17 14:27:34.899408 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:34.899122    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d882585-6a65-4e36-b1c7-7ec4562cd182-credential-socket\") pod \"e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd\" (UID: \"9d882585-6a65-4e36-b1c7-7ec4562cd182\") " pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd"
Sep 17 14:27:34.899408 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:34.899187    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:34.899408 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:34.899282    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d882585-6a65-4e36-b1c7-7ec4562cd182-istio-envoy\") pod \"e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd\" (UID: \"9d882585-6a65-4e36-b1c7-7ec4562cd182\") " pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd"
Sep 17 14:27:34.899408 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:34.899335    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvbz8\" (UniqueName: \"kubernetes.io/projected/9d882585-6a65-4e36-b1c7-7ec4562cd182-kube-api-access-dvbz8\") pod \"e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd\" (UID: \"9d882585-6a65-4e36-b1c7-7ec4562cd182\") " pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd"
Sep 17 14:27:34.900078 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:34.899435    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d882585-6a65-4e36-b1c7-7ec4562cd182-istio-data\") pod \"e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd\" (UID: \"9d882585-6a65-4e36-b1c7-7ec4562cd182\") " pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd"
Sep 17 14:27:35.000624 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:35.000560    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d882585-6a65-4e36-b1c7-7ec4562cd182-workload-socket\") pod \"e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd\" (UID: \"9d882585-6a65-4e36-b1c7-7ec4562cd182\") " pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd"
Sep 17 14:27:35.001012 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:35.000645    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d882585-6a65-4e36-b1c7-7ec4562cd182-istio-podinfo\") pod \"e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd\" (UID: \"9d882585-6a65-4e36-b1c7-7ec4562cd182\") " pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd"
Sep 17 14:27:35.001012 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:35.000691    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d882585-6a65-4e36-b1c7-7ec4562cd182-istio-token\") pod \"e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd\" (UID: \"9d882585-6a65-4e36-b1c7-7ec4562cd182\") " pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd"
Sep 17 14:27:35.001012 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:35.000708    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d882585-6a65-4e36-b1c7-7ec4562cd182-credential-socket\") pod \"e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd\" (UID: \"9d882585-6a65-4e36-b1c7-7ec4562cd182\") " pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd"
Sep 17 14:27:35.001012 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:35.000747    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d882585-6a65-4e36-b1c7-7ec4562cd182-istio-envoy\") pod \"e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd\" (UID: \"9d882585-6a65-4e36-b1c7-7ec4562cd182\") " pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd"
Sep 17 14:27:35.001012 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:35.000766    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvbz8\" (UniqueName: \"kubernetes.io/projected/9d882585-6a65-4e36-b1c7-7ec4562cd182-kube-api-access-dvbz8\") pod \"e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd\" (UID: \"9d882585-6a65-4e36-b1c7-7ec4562cd182\") " pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd"
Sep 17 14:27:35.001012 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:35.000790    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d882585-6a65-4e36-b1c7-7ec4562cd182-istio-data\") pod \"e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd\" (UID: \"9d882585-6a65-4e36-b1c7-7ec4562cd182\") " pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd"
Sep 17 14:27:35.001012 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:35.000825    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d882585-6a65-4e36-b1c7-7ec4562cd182-workload-certs\") pod \"e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd\" (UID: \"9d882585-6a65-4e36-b1c7-7ec4562cd182\") " pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd"
Sep 17 14:27:35.001012 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:35.000844    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d882585-6a65-4e36-b1c7-7ec4562cd182-istiod-ca-cert\") pod \"e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd\" (UID: \"9d882585-6a65-4e36-b1c7-7ec4562cd182\") " pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd"
Sep 17 14:27:35.001513 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:35.001493    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d882585-6a65-4e36-b1c7-7ec4562cd182-istiod-ca-cert\") pod \"e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd\" (UID: \"9d882585-6a65-4e36-b1c7-7ec4562cd182\") " pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd"
Sep 17 14:27:35.001743 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:35.001727    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d882585-6a65-4e36-b1c7-7ec4562cd182-workload-socket\") pod \"e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd\" (UID: \"9d882585-6a65-4e36-b1c7-7ec4562cd182\") " pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd"
Sep 17 14:27:35.004100 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:35.004063    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d882585-6a65-4e36-b1c7-7ec4562cd182-istio-podinfo\") pod \"e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd\" (UID: \"9d882585-6a65-4e36-b1c7-7ec4562cd182\") " pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd"
Sep 17 14:27:35.004242 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:35.004119    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d882585-6a65-4e36-b1c7-7ec4562cd182-istio-envoy\") pod \"e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd\" (UID: \"9d882585-6a65-4e36-b1c7-7ec4562cd182\") " pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd"
Sep 17 14:27:35.004498 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:35.004476    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d882585-6a65-4e36-b1c7-7ec4562cd182-credential-socket\") pod \"e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd\" (UID: \"9d882585-6a65-4e36-b1c7-7ec4562cd182\") " pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd"
Sep 17 14:27:35.004593 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:35.004566    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d882585-6a65-4e36-b1c7-7ec4562cd182-istio-data\") pod \"e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd\" (UID: \"9d882585-6a65-4e36-b1c7-7ec4562cd182\") " pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd"
Sep 17 14:27:35.004762 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:35.004744    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d882585-6a65-4e36-b1c7-7ec4562cd182-workload-certs\") pod \"e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd\" (UID: \"9d882585-6a65-4e36-b1c7-7ec4562cd182\") " pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd"
Sep 17 14:27:35.011809 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:35.011786    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d882585-6a65-4e36-b1c7-7ec4562cd182-istio-token\") pod \"e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd\" (UID: \"9d882585-6a65-4e36-b1c7-7ec4562cd182\") " pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd"
Sep 17 14:27:35.013834 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:35.013806    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvbz8\" (UniqueName: \"kubernetes.io/projected/9d882585-6a65-4e36-b1c7-7ec4562cd182-kube-api-access-dvbz8\") pod \"e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd\" (UID: \"9d882585-6a65-4e36-b1c7-7ec4562cd182\") " pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd"
Sep 17 14:27:35.089013 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:35.088977    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd"
Sep 17 14:27:35.398834 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:35.398677    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:35.400647 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:35.400622    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd"]
Sep 17 14:27:35.401752 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:27:35.401727    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d882585_6a65_4e36_b1c7_7ec4562cd182.slice/crio-a247dd8efdf7c88c056642a1388ffc0b47bae78628043b44f034978d83dbe99b WatchSource:0}: Error finding container a247dd8efdf7c88c056642a1388ffc0b47bae78628043b44f034978d83dbe99b: Status 404 returned error can't find the container with id a247dd8efdf7c88c056642a1388ffc0b47bae78628043b44f034978d83dbe99b
Sep 17 14:27:35.404434 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:35.404402    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:27:35.404524 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:35.404480    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:27:35.404587 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:35.404522    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:27:35.583295 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:35.583258    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:35.677406 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:35.677310    2582 patch_prober.go:28] interesting pod/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Sep 17 14:27:35.677406 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:35.677371    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr" podUID="7d7bc84f-6345-4a14-a73b-21e0aa0a30ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Sep 17 14:27:36.264246 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:36.264206    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:36.380384 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:36.380350    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd" event={"ID":"9d882585-6a65-4e36-b1c7-7ec4562cd182","Type":"ContainerStarted","Data":"c9cb969d5b97ef6679223ed8a0aa15f66d5d9d381a8d3b5b7d14a25c9252516e"}
Sep 17 14:27:36.380581 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:36.380401    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd" event={"ID":"9d882585-6a65-4e36-b1c7-7ec4562cd182","Type":"ContainerStarted","Data":"a247dd8efdf7c88c056642a1388ffc0b47bae78628043b44f034978d83dbe99b"}
Sep 17 14:27:36.402459 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:36.402412    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd" podStartSLOduration=2.402392823 podStartE2EDuration="2.402392823s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:27:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:27:36.400272665 +0000 UTC m=+1357.401816140" watchObservedRunningTime="2026-09-17 14:27:36.402392823 +0000 UTC m=+1357.403936298"
Sep 17 14:27:36.677117 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:36.677088    2582 patch_prober.go:28] interesting pod/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Sep 17 14:27:36.677257 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:36.677140    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr" podUID="7d7bc84f-6345-4a14-a73b-21e0aa0a30ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Sep 17 14:27:37.089183 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:37.089141    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd"
Sep 17 14:27:37.094410 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:37.094382    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd"
Sep 17 14:27:37.386144 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:37.386050    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd"
Sep 17 14:27:37.387025 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:37.387007    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd"
Sep 17 14:27:37.632726 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:37.632690    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:37.676891 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:37.676813    2582 patch_prober.go:28] interesting pod/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Sep 17 14:27:37.676891 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:37.676873    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr" podUID="7d7bc84f-6345-4a14-a73b-21e0aa0a30ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Sep 17 14:27:38.313682 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:38.313642    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:38.435330 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:38.435295    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:38.677122 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:38.677029    2582 patch_prober.go:28] interesting pod/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Sep 17 14:27:38.677122 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:38.677105    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr" podUID="7d7bc84f-6345-4a14-a73b-21e0aa0a30ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Sep 17 14:27:39.194256 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:39.194223    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr"
Sep 17 14:27:39.250199 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:39.250163    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:39.681813 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:39.681721    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr"
Sep 17 14:27:39.682784 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:39.682767    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr"
Sep 17 14:27:40.012299 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:40.012259    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:40.666221 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:40.666163    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:41.429025 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:41.428967    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:42.210015 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:42.209980    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:42.308425 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:42.308381    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:42.488182 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:42.488073    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:42.687813 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:42.687761    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:42.791781 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:42.791668    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:43.158918 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:43.158832    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:43.854448 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:43.854418    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:44.769570 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:44.769533    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:45.398698 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:45.398667    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:45.499074 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:45.499037    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:45.601004 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:45.600932    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:45.611181 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:45.611143    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a095586-6k7cv"]
Sep 17 14:27:45.613530 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:45.613508    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a095586-6k7cv"]
Sep 17 14:27:45.729372 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:45.729313    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:46.050509 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:46.050414    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:46.701207 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:46.701165    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:47.493884 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:47.493851    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52beca38-a48f-4292-b94a-92671dc121bc" path="/var/lib/kubelet/pods/52beca38-a48f-4292-b94a-92671dc121bc/volumes"
Sep 17 14:27:47.580584 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:47.580552    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:48.349940 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:48.349904    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:48.840040 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:48.840002    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:49.092728 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:49.092597    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:49.737057 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:49.737021    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:50.421597 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:50.421559    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:51.734722 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:51.734637    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:52.653940 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:52.653902    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:53.092295 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:53.092249    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:53.367280 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:53.367197    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:53.816260 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:53.816223    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-1a095586/e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7"]
Sep 17 14:27:53.919443 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:53.919404    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:54.087950 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:54.087863    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:54.839202 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:54.839165    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:55.586811 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:55.586780    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:56.356451 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:56.356419    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:27:59.536639 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:59.536584    2582 generic.go:356] "Generic (PLEG): container finished" podID="9d882585-6a65-4e36-b1c7-7ec4562cd182" containerID="c9cb969d5b97ef6679223ed8a0aa15f66d5d9d381a8d3b5b7d14a25c9252516e" exitCode=0
Sep 17 14:27:59.537106 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:59.536640    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd" event={"ID":"9d882585-6a65-4e36-b1c7-7ec4562cd182","Type":"ContainerDied","Data":"c9cb969d5b97ef6679223ed8a0aa15f66d5d9d381a8d3b5b7d14a25c9252516e"}
Sep 17 14:27:59.537224 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:59.537206    2582 scope.go:117] "RemoveContainer" containerID="c9cb969d5b97ef6679223ed8a0aa15f66d5d9d381a8d3b5b7d14a25c9252516e"
Sep 17 14:27:59.538500 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:59.538464    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:27:59.538647 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:59.538550    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:27:59.538647 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:59.538600    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:27:59.683042 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:27:59.683009    2582 scope.go:117] "RemoveContainer" containerID="76c92fa7d1c81ab9f4d11d51fb6d7590b59a4ef4a1293187ca2a472efee17ab8"
Sep 17 14:28:00.551192 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:00.551147    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd" event={"ID":"9d882585-6a65-4e36-b1c7-7ec4562cd182","Type":"ContainerStarted","Data":"31c892e1a703191ff331e65bb3aee00153a112911ffb915aa9bc0efe0513cc9a"}
Sep 17 14:28:01.089799 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:01.089764    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd"
Sep 17 14:28:01.089963 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:01.089928    2582 patch_prober.go:28] interesting pod/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Sep 17 14:28:01.090008 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:01.089973    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd" podUID="9d882585-6a65-4e36-b1c7-7ec4562cd182" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Sep 17 14:28:01.161209 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:01.161174    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:28:02.089849 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:02.089811    2582 patch_prober.go:28] interesting pod/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Sep 17 14:28:02.090306 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:02.089898    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd" podUID="9d882585-6a65-4e36-b1c7-7ec4562cd182" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Sep 17 14:28:03.089985 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:03.089951    2582 patch_prober.go:28] interesting pod/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Sep 17 14:28:03.090373 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:03.090015    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd" podUID="9d882585-6a65-4e36-b1c7-7ec4562cd182" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Sep 17 14:28:04.089504 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:04.089470    2582 patch_prober.go:28] interesting pod/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Sep 17 14:28:04.089721 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:04.089524    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd" podUID="9d882585-6a65-4e36-b1c7-7ec4562cd182" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Sep 17 14:28:05.090177 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:05.090132    2582 patch_prober.go:28] interesting pod/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Sep 17 14:28:05.090568 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:05.090207    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd" podUID="9d882585-6a65-4e36-b1c7-7ec4562cd182" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Sep 17 14:28:06.095281 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:06.095247    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd"
Sep 17 14:28:06.095751 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:06.095725    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd"
Sep 17 14:28:06.096461 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:06.096443    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd"
Sep 17 14:28:08.864876 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:08.864828    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-1a095586/e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7" podUID="182c2e69-c5c8-41e8-8190-a8bc6ca56cf2" containerName="main" containerID="cri-o://eec7514c641da72913d4faf2f08ffc0f318b2fff11fbee30853586e0affad02d" gracePeriod=45
Sep 17 14:28:09.121658 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:09.121574    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-1a095586/e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7"
Sep 17 14:28:09.220257 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:09.220224    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/182c2e69-c5c8-41e8-8190-a8bc6ca56cf2-home\") pod \"182c2e69-c5c8-41e8-8190-a8bc6ca56cf2\" (UID: \"182c2e69-c5c8-41e8-8190-a8bc6ca56cf2\") "
Sep 17 14:28:09.220466 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:09.220288    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/182c2e69-c5c8-41e8-8190-a8bc6ca56cf2-tmp-dir\") pod \"182c2e69-c5c8-41e8-8190-a8bc6ca56cf2\" (UID: \"182c2e69-c5c8-41e8-8190-a8bc6ca56cf2\") "
Sep 17 14:28:09.220466 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:09.220362    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gn49s\" (UniqueName: \"kubernetes.io/projected/182c2e69-c5c8-41e8-8190-a8bc6ca56cf2-kube-api-access-gn49s\") pod \"182c2e69-c5c8-41e8-8190-a8bc6ca56cf2\" (UID: \"182c2e69-c5c8-41e8-8190-a8bc6ca56cf2\") "
Sep 17 14:28:09.220466 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:09.220390    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/182c2e69-c5c8-41e8-8190-a8bc6ca56cf2-dshm\") pod \"182c2e69-c5c8-41e8-8190-a8bc6ca56cf2\" (UID: \"182c2e69-c5c8-41e8-8190-a8bc6ca56cf2\") "
Sep 17 14:28:09.220466 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:09.220435    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/182c2e69-c5c8-41e8-8190-a8bc6ca56cf2-tls-certs\") pod \"182c2e69-c5c8-41e8-8190-a8bc6ca56cf2\" (UID: \"182c2e69-c5c8-41e8-8190-a8bc6ca56cf2\") "
Sep 17 14:28:09.220657 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:09.220468    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/182c2e69-c5c8-41e8-8190-a8bc6ca56cf2-model-cache\") pod \"182c2e69-c5c8-41e8-8190-a8bc6ca56cf2\" (UID: \"182c2e69-c5c8-41e8-8190-a8bc6ca56cf2\") "
Sep 17 14:28:09.220657 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:09.220521    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/182c2e69-c5c8-41e8-8190-a8bc6ca56cf2-home" (OuterVolumeSpecName: "home") pod "182c2e69-c5c8-41e8-8190-a8bc6ca56cf2" (UID: "182c2e69-c5c8-41e8-8190-a8bc6ca56cf2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:28:09.220657 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:09.220536    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/182c2e69-c5c8-41e8-8190-a8bc6ca56cf2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "182c2e69-c5c8-41e8-8190-a8bc6ca56cf2" (UID: "182c2e69-c5c8-41e8-8190-a8bc6ca56cf2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:28:09.220829 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:09.220807    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/182c2e69-c5c8-41e8-8190-a8bc6ca56cf2-model-cache" (OuterVolumeSpecName: "model-cache") pod "182c2e69-c5c8-41e8-8190-a8bc6ca56cf2" (UID: "182c2e69-c5c8-41e8-8190-a8bc6ca56cf2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:28:09.220968 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:09.220953    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/182c2e69-c5c8-41e8-8190-a8bc6ca56cf2-model-cache\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:28:09.221036 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:09.220971    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/182c2e69-c5c8-41e8-8190-a8bc6ca56cf2-home\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:28:09.221036 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:09.220983    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/182c2e69-c5c8-41e8-8190-a8bc6ca56cf2-tmp-dir\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:28:09.222652 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:09.222626    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/182c2e69-c5c8-41e8-8190-a8bc6ca56cf2-dshm" (OuterVolumeSpecName: "dshm") pod "182c2e69-c5c8-41e8-8190-a8bc6ca56cf2" (UID: "182c2e69-c5c8-41e8-8190-a8bc6ca56cf2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:28:09.222757 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:09.222661    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/182c2e69-c5c8-41e8-8190-a8bc6ca56cf2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "182c2e69-c5c8-41e8-8190-a8bc6ca56cf2" (UID: "182c2e69-c5c8-41e8-8190-a8bc6ca56cf2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:28:09.222798 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:09.222757    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/182c2e69-c5c8-41e8-8190-a8bc6ca56cf2-kube-api-access-gn49s" (OuterVolumeSpecName: "kube-api-access-gn49s") pod "182c2e69-c5c8-41e8-8190-a8bc6ca56cf2" (UID: "182c2e69-c5c8-41e8-8190-a8bc6ca56cf2"). InnerVolumeSpecName "kube-api-access-gn49s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:28:09.322430 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:09.322390    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gn49s\" (UniqueName: \"kubernetes.io/projected/182c2e69-c5c8-41e8-8190-a8bc6ca56cf2-kube-api-access-gn49s\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:28:09.322430 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:09.322425    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/182c2e69-c5c8-41e8-8190-a8bc6ca56cf2-dshm\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:28:09.322430 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:09.322438    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/182c2e69-c5c8-41e8-8190-a8bc6ca56cf2-tls-certs\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:28:09.611557 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:09.611521    2582 generic.go:356] "Generic (PLEG): container finished" podID="182c2e69-c5c8-41e8-8190-a8bc6ca56cf2" containerID="eec7514c641da72913d4faf2f08ffc0f318b2fff11fbee30853586e0affad02d" exitCode=0
Sep 17 14:28:09.611808 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:09.611567    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-1a095586/e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7"
Sep 17 14:28:09.611808 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:09.611637    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-1a095586/e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7" event={"ID":"182c2e69-c5c8-41e8-8190-a8bc6ca56cf2","Type":"ContainerDied","Data":"eec7514c641da72913d4faf2f08ffc0f318b2fff11fbee30853586e0affad02d"}
Sep 17 14:28:09.611808 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:09.611686    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-1a095586/e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7" event={"ID":"182c2e69-c5c8-41e8-8190-a8bc6ca56cf2","Type":"ContainerDied","Data":"db9ebd728d779211f5b939f2adcb6c8d7c20fdbbc8660acf438f66656a1a838f"}
Sep 17 14:28:09.611808 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:09.611704    2582 scope.go:117] "RemoveContainer" containerID="eec7514c641da72913d4faf2f08ffc0f318b2fff11fbee30853586e0affad02d"
Sep 17 14:28:09.621293 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:09.621277    2582 scope.go:117] "RemoveContainer" containerID="eec7514c641da72913d4faf2f08ffc0f318b2fff11fbee30853586e0affad02d"
Sep 17 14:28:09.621559 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:28:09.621541    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eec7514c641da72913d4faf2f08ffc0f318b2fff11fbee30853586e0affad02d\": container with ID starting with eec7514c641da72913d4faf2f08ffc0f318b2fff11fbee30853586e0affad02d not found: ID does not exist" containerID="eec7514c641da72913d4faf2f08ffc0f318b2fff11fbee30853586e0affad02d"
Sep 17 14:28:09.621602 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:09.621568    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eec7514c641da72913d4faf2f08ffc0f318b2fff11fbee30853586e0affad02d"} err="failed to get container status \"eec7514c641da72913d4faf2f08ffc0f318b2fff11fbee30853586e0affad02d\": rpc error: code = NotFound desc = could not find container \"eec7514c641da72913d4faf2f08ffc0f318b2fff11fbee30853586e0affad02d\": container with ID starting with eec7514c641da72913d4faf2f08ffc0f318b2fff11fbee30853586e0affad02d not found: ID does not exist"
Sep 17 14:28:09.654399 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:09.654363    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-1a095586/e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7"]
Sep 17 14:28:09.659105 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:09.659074    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-1a095586/e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7"]
Sep 17 14:28:11.494886 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:11.494850    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="182c2e69-c5c8-41e8-8190-a8bc6ca56cf2" path="/var/lib/kubelet/pods/182c2e69-c5c8-41e8-8190-a8bc6ca56cf2/volumes"
Sep 17 14:28:14.058067 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:14.058015    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-1a095586/e2e-lifecycle-model-1a095586-kserve-7d9c8466c9-rrwj7" podUID="182c2e69-c5c8-41e8-8190-a8bc6ca56cf2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Sep 17 14:28:24.371575 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:24.371537    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-cc3801/auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9"]
Sep 17 14:28:24.372019 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:24.372007    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="182c2e69-c5c8-41e8-8190-a8bc6ca56cf2" containerName="main"
Sep 17 14:28:24.372019 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:24.372020    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="182c2e69-c5c8-41e8-8190-a8bc6ca56cf2" containerName="main"
Sep 17 14:28:24.372120 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:24.372112    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="182c2e69-c5c8-41e8-8190-a8bc6ca56cf2" containerName="main"
Sep 17 14:28:24.374401 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:24.374386    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-cc3801/auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9"
Sep 17 14:28:24.377855 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:24.377828    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-cc3801\"/\"default-dockercfg-h86g7\""
Sep 17 14:28:24.378008 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:24.377850    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-cc3801\"/\"openshift-service-ca.crt\""
Sep 17 14:28:24.378008 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:24.377888    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-cc3801\"/\"kube-root-ca.crt\""
Sep 17 14:28:24.378008 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:24.377896    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-cc3801\"/\"auth-test-model-cc3801-kserve-self-signed-certs\""
Sep 17 14:28:24.385425 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:24.385401    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-cc3801/auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9"]
Sep 17 14:28:24.489847 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:24.489807    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d2f89a0-b1b2-47d9-976a-2435cf66750a-tmp-dir\") pod \"auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9\" (UID: \"4d2f89a0-b1b2-47d9-976a-2435cf66750a\") " pod="ai-tenant-e2e-shared-a-w3-cc3801/auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9"
Sep 17 14:28:24.489847 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:24.489847    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p778h\" (UniqueName: \"kubernetes.io/projected/4d2f89a0-b1b2-47d9-976a-2435cf66750a-kube-api-access-p778h\") pod \"auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9\" (UID: \"4d2f89a0-b1b2-47d9-976a-2435cf66750a\") " pod="ai-tenant-e2e-shared-a-w3-cc3801/auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9"
Sep 17 14:28:24.490118 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:24.489975    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d2f89a0-b1b2-47d9-976a-2435cf66750a-tls-certs\") pod \"auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9\" (UID: \"4d2f89a0-b1b2-47d9-976a-2435cf66750a\") " pod="ai-tenant-e2e-shared-a-w3-cc3801/auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9"
Sep 17 14:28:24.490118 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:24.490018    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d2f89a0-b1b2-47d9-976a-2435cf66750a-model-cache\") pod \"auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9\" (UID: \"4d2f89a0-b1b2-47d9-976a-2435cf66750a\") " pod="ai-tenant-e2e-shared-a-w3-cc3801/auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9"
Sep 17 14:28:24.490118 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:24.490065    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d2f89a0-b1b2-47d9-976a-2435cf66750a-dshm\") pod \"auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9\" (UID: \"4d2f89a0-b1b2-47d9-976a-2435cf66750a\") " pod="ai-tenant-e2e-shared-a-w3-cc3801/auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9"
Sep 17 14:28:24.490118 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:24.490113    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d2f89a0-b1b2-47d9-976a-2435cf66750a-home\") pod \"auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9\" (UID: \"4d2f89a0-b1b2-47d9-976a-2435cf66750a\") " pod="ai-tenant-e2e-shared-a-w3-cc3801/auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9"
Sep 17 14:28:24.591188 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:24.591138    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d2f89a0-b1b2-47d9-976a-2435cf66750a-tls-certs\") pod \"auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9\" (UID: \"4d2f89a0-b1b2-47d9-976a-2435cf66750a\") " pod="ai-tenant-e2e-shared-a-w3-cc3801/auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9"
Sep 17 14:28:24.591384 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:24.591194    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d2f89a0-b1b2-47d9-976a-2435cf66750a-model-cache\") pod \"auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9\" (UID: \"4d2f89a0-b1b2-47d9-976a-2435cf66750a\") " pod="ai-tenant-e2e-shared-a-w3-cc3801/auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9"
Sep 17 14:28:24.591384 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:24.591228    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d2f89a0-b1b2-47d9-976a-2435cf66750a-dshm\") pod \"auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9\" (UID: \"4d2f89a0-b1b2-47d9-976a-2435cf66750a\") " pod="ai-tenant-e2e-shared-a-w3-cc3801/auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9"
Sep 17 14:28:24.591384 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:24.591374    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d2f89a0-b1b2-47d9-976a-2435cf66750a-home\") pod \"auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9\" (UID: \"4d2f89a0-b1b2-47d9-976a-2435cf66750a\") " pod="ai-tenant-e2e-shared-a-w3-cc3801/auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9"
Sep 17 14:28:24.591561 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:24.591511    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d2f89a0-b1b2-47d9-976a-2435cf66750a-tmp-dir\") pod \"auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9\" (UID: \"4d2f89a0-b1b2-47d9-976a-2435cf66750a\") " pod="ai-tenant-e2e-shared-a-w3-cc3801/auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9"
Sep 17 14:28:24.591561 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:24.591546    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p778h\" (UniqueName: \"kubernetes.io/projected/4d2f89a0-b1b2-47d9-976a-2435cf66750a-kube-api-access-p778h\") pod \"auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9\" (UID: \"4d2f89a0-b1b2-47d9-976a-2435cf66750a\") " pod="ai-tenant-e2e-shared-a-w3-cc3801/auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9"
Sep 17 14:28:24.591715 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:24.591692    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d2f89a0-b1b2-47d9-976a-2435cf66750a-model-cache\") pod \"auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9\" (UID: \"4d2f89a0-b1b2-47d9-976a-2435cf66750a\") " pod="ai-tenant-e2e-shared-a-w3-cc3801/auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9"
Sep 17 14:28:24.592112 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:24.592080    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d2f89a0-b1b2-47d9-976a-2435cf66750a-home\") pod \"auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9\" (UID: \"4d2f89a0-b1b2-47d9-976a-2435cf66750a\") " pod="ai-tenant-e2e-shared-a-w3-cc3801/auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9"
Sep 17 14:28:24.592247 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:24.592194    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d2f89a0-b1b2-47d9-976a-2435cf66750a-tmp-dir\") pod \"auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9\" (UID: \"4d2f89a0-b1b2-47d9-976a-2435cf66750a\") " pod="ai-tenant-e2e-shared-a-w3-cc3801/auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9"
Sep 17 14:28:24.592247 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:24.592176    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:28:24.593996 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:24.593967    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d2f89a0-b1b2-47d9-976a-2435cf66750a-dshm\") pod \"auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9\" (UID: \"4d2f89a0-b1b2-47d9-976a-2435cf66750a\") " pod="ai-tenant-e2e-shared-a-w3-cc3801/auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9"
Sep 17 14:28:24.594430 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:24.594411    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d2f89a0-b1b2-47d9-976a-2435cf66750a-tls-certs\") pod \"auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9\" (UID: \"4d2f89a0-b1b2-47d9-976a-2435cf66750a\") " pod="ai-tenant-e2e-shared-a-w3-cc3801/auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9"
Sep 17 14:28:24.602826 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:24.602796    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p778h\" (UniqueName: \"kubernetes.io/projected/4d2f89a0-b1b2-47d9-976a-2435cf66750a-kube-api-access-p778h\") pod \"auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9\" (UID: \"4d2f89a0-b1b2-47d9-976a-2435cf66750a\") " pod="ai-tenant-e2e-shared-a-w3-cc3801/auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9"
Sep 17 14:28:24.684868 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:24.684777    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-cc3801/auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9"
Sep 17 14:28:24.973155 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:24.973129    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-cc3801/auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9"]
Sep 17 14:28:24.975230 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:28:24.975205    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4d2f89a0_b1b2_47d9_976a_2435cf66750a.slice/crio-005189e1074a97990be3c5c190d114590ba0ae8250583a3143b3b4325b798c5c WatchSource:0}: Error finding container 005189e1074a97990be3c5c190d114590ba0ae8250583a3143b3b4325b798c5c: Status 404 returned error can't find the container with id 005189e1074a97990be3c5c190d114590ba0ae8250583a3143b3b4325b798c5c
Sep 17 14:28:25.726748 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:25.726708    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-cc3801/auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9" event={"ID":"4d2f89a0-b1b2-47d9-976a-2435cf66750a","Type":"ContainerStarted","Data":"54867acb7e8999cd1c702c44d13ca482a98434bb4acd7df85ef7387d30440a4a"}
Sep 17 14:28:25.727332 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:25.727292    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-cc3801/auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9" event={"ID":"4d2f89a0-b1b2-47d9-976a-2435cf66750a","Type":"ContainerStarted","Data":"005189e1074a97990be3c5c190d114590ba0ae8250583a3143b3b4325b798c5c"}
Sep 17 14:28:25.747198 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:25.747144    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-cc3801/auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9" podStartSLOduration=1.7471298659999999 podStartE2EDuration="1.747129866s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:28:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:28:25.744528278 +0000 UTC m=+1406.746071788" watchObservedRunningTime="2026-09-17 14:28:25.747129866 +0000 UTC m=+1406.748673386"
Sep 17 14:28:27.574211 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:27.574173    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-b-9497140e-kserve-794df4446b-stw2n"]
Sep 17 14:28:27.599354 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:27.599321    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:28:30.934408 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:30.934344    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:28:31.295687 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:31.295651    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:28:34.685395 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:34.685361    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-cc3801/auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9"
Sep 17 14:28:34.685838 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:34.685644    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-cc3801/auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9"
Sep 17 14:28:34.698331 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:34.698304    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-cc3801/auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9"
Sep 17 14:28:34.793682 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:34.793645    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-cc3801/auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9"
Sep 17 14:28:36.856903 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:36.856872    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:28:42.595593 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:42.595545    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-b-9497140e-kserve-794df4446b-stw2n" podUID="b259eb30-d5e7-4fde-b353-9dbcac62ece9" containerName="main" containerID="cri-o://f0012f882a2fe4cd8f77b4d57cf8d6e18da465f5197f638964cc716bcf363bfb" gracePeriod=45
Sep 17 14:28:42.832348 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:42.832306    2582 generic.go:356] "Generic (PLEG): container finished" podID="b259eb30-d5e7-4fde-b353-9dbcac62ece9" containerID="f0012f882a2fe4cd8f77b4d57cf8d6e18da465f5197f638964cc716bcf363bfb" exitCode=0
Sep 17 14:28:42.832545 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:42.832384    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-b-9497140e-kserve-794df4446b-stw2n" event={"ID":"b259eb30-d5e7-4fde-b353-9dbcac62ece9","Type":"ContainerDied","Data":"f0012f882a2fe4cd8f77b4d57cf8d6e18da465f5197f638964cc716bcf363bfb"}
Sep 17 14:28:42.864590 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:42.864562    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-b-9497140e-kserve-794df4446b-stw2n"
Sep 17 14:28:42.907081 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:42.907049    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b259eb30-d5e7-4fde-b353-9dbcac62ece9-dshm\") pod \"b259eb30-d5e7-4fde-b353-9dbcac62ece9\" (UID: \"b259eb30-d5e7-4fde-b353-9dbcac62ece9\") "
Sep 17 14:28:42.907248 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:42.907107    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b259eb30-d5e7-4fde-b353-9dbcac62ece9-tmp-dir\") pod \"b259eb30-d5e7-4fde-b353-9dbcac62ece9\" (UID: \"b259eb30-d5e7-4fde-b353-9dbcac62ece9\") "
Sep 17 14:28:42.907248 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:42.907162    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9jbww\" (UniqueName: \"kubernetes.io/projected/b259eb30-d5e7-4fde-b353-9dbcac62ece9-kube-api-access-9jbww\") pod \"b259eb30-d5e7-4fde-b353-9dbcac62ece9\" (UID: \"b259eb30-d5e7-4fde-b353-9dbcac62ece9\") "
Sep 17 14:28:42.907248 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:42.907200    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b259eb30-d5e7-4fde-b353-9dbcac62ece9-tls-certs\") pod \"b259eb30-d5e7-4fde-b353-9dbcac62ece9\" (UID: \"b259eb30-d5e7-4fde-b353-9dbcac62ece9\") "
Sep 17 14:28:42.907415 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:42.907258    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b259eb30-d5e7-4fde-b353-9dbcac62ece9-home\") pod \"b259eb30-d5e7-4fde-b353-9dbcac62ece9\" (UID: \"b259eb30-d5e7-4fde-b353-9dbcac62ece9\") "
Sep 17 14:28:42.907415 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:42.907302    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b259eb30-d5e7-4fde-b353-9dbcac62ece9-model-cache\") pod \"b259eb30-d5e7-4fde-b353-9dbcac62ece9\" (UID: \"b259eb30-d5e7-4fde-b353-9dbcac62ece9\") "
Sep 17 14:28:42.907524 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:42.907444    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b259eb30-d5e7-4fde-b353-9dbcac62ece9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b259eb30-d5e7-4fde-b353-9dbcac62ece9" (UID: "b259eb30-d5e7-4fde-b353-9dbcac62ece9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:28:42.907795 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:42.907664    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b259eb30-d5e7-4fde-b353-9dbcac62ece9-home" (OuterVolumeSpecName: "home") pod "b259eb30-d5e7-4fde-b353-9dbcac62ece9" (UID: "b259eb30-d5e7-4fde-b353-9dbcac62ece9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:28:42.907921 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:42.907838    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b259eb30-d5e7-4fde-b353-9dbcac62ece9-model-cache" (OuterVolumeSpecName: "model-cache") pod "b259eb30-d5e7-4fde-b353-9dbcac62ece9" (UID: "b259eb30-d5e7-4fde-b353-9dbcac62ece9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:28:42.907921 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:42.907881    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b259eb30-d5e7-4fde-b353-9dbcac62ece9-tmp-dir\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:28:42.907921 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:42.907898    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b259eb30-d5e7-4fde-b353-9dbcac62ece9-home\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:28:42.909556 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:42.909522    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b259eb30-d5e7-4fde-b353-9dbcac62ece9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b259eb30-d5e7-4fde-b353-9dbcac62ece9" (UID: "b259eb30-d5e7-4fde-b353-9dbcac62ece9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:28:42.909791 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:42.909772    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b259eb30-d5e7-4fde-b353-9dbcac62ece9-dshm" (OuterVolumeSpecName: "dshm") pod "b259eb30-d5e7-4fde-b353-9dbcac62ece9" (UID: "b259eb30-d5e7-4fde-b353-9dbcac62ece9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:28:42.909856 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:42.909801    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b259eb30-d5e7-4fde-b353-9dbcac62ece9-kube-api-access-9jbww" (OuterVolumeSpecName: "kube-api-access-9jbww") pod "b259eb30-d5e7-4fde-b353-9dbcac62ece9" (UID: "b259eb30-d5e7-4fde-b353-9dbcac62ece9"). InnerVolumeSpecName "kube-api-access-9jbww". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:28:43.008883 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:43.008835    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b259eb30-d5e7-4fde-b353-9dbcac62ece9-model-cache\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:28:43.008883 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:43.008874    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b259eb30-d5e7-4fde-b353-9dbcac62ece9-dshm\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:28:43.008883 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:43.008885    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jbww\" (UniqueName: \"kubernetes.io/projected/b259eb30-d5e7-4fde-b353-9dbcac62ece9-kube-api-access-9jbww\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:28:43.009127 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:43.008897    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b259eb30-d5e7-4fde-b353-9dbcac62ece9-tls-certs\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:28:43.841049 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:43.840956    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-b-9497140e-kserve-794df4446b-stw2n"
Sep 17 14:28:43.841049 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:43.840996    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-b-9497140e-kserve-794df4446b-stw2n" event={"ID":"b259eb30-d5e7-4fde-b353-9dbcac62ece9","Type":"ContainerDied","Data":"d2f65379ba3603eb726134e2a8462b533508e5c7f0cadb45ffcdd914b349f8b8"}
Sep 17 14:28:43.841520 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:43.841061    2582 scope.go:117] "RemoveContainer" containerID="f0012f882a2fe4cd8f77b4d57cf8d6e18da465f5197f638964cc716bcf363bfb"
Sep 17 14:28:43.859069 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:43.859043    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-b-9497140e-kserve-794df4446b-stw2n"]
Sep 17 14:28:43.864954 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:43.864929    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-b-9497140e-kserve-794df4446b-stw2n"]
Sep 17 14:28:45.494400 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:45.494363    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b259eb30-d5e7-4fde-b353-9dbcac62ece9" path="/var/lib/kubelet/pods/b259eb30-d5e7-4fde-b353-9dbcac62ece9/volumes"
Sep 17 14:28:46.634510 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:46.634469    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46"]
Sep 17 14:28:46.702801 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:46.702769    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:28:51.325146 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:51.325115    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:28:51.425194 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:51.425155    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:28:51.548956 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:51.548917    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn"]
Sep 17 14:28:51.549194 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:51.549163    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn" podUID="3ab0e2f8-287d-4ba7-b06c-e3bef8457239" containerName="istio-proxy" containerID="cri-o://924ff6cafa7b48480c3d85b6ec4e75b58a864e0b94bbd54637650dc887286726" gracePeriod=30
Sep 17 14:28:51.571237 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:51.571200    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:28:52.893243 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:52.893158    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:28:53.036173 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:53.036136    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:28:53.766045 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:53.766007    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:28:53.864371 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:53.864334    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:28:54.011765 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:54.011731    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:28:55.218345 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:55.218301    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:28:56.808570 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:56.808540    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn"
Sep 17 14:28:56.923459 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:56.923365    2582 generic.go:356] "Generic (PLEG): container finished" podID="3ab0e2f8-287d-4ba7-b06c-e3bef8457239" containerID="924ff6cafa7b48480c3d85b6ec4e75b58a864e0b94bbd54637650dc887286726" exitCode=0
Sep 17 14:28:56.923459 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:56.923415    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn"
Sep 17 14:28:56.923459 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:56.923447    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn" event={"ID":"3ab0e2f8-287d-4ba7-b06c-e3bef8457239","Type":"ContainerDied","Data":"924ff6cafa7b48480c3d85b6ec4e75b58a864e0b94bbd54637650dc887286726"}
Sep 17 14:28:56.923764 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:56.923530    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn" event={"ID":"3ab0e2f8-287d-4ba7-b06c-e3bef8457239","Type":"ContainerDied","Data":"cecc4a52693e4cfbf0fcad202d6aa367456a3c2de5debcd4242dd9cc421017bf"}
Sep 17 14:28:56.923764 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:56.923554    2582 scope.go:117] "RemoveContainer" containerID="924ff6cafa7b48480c3d85b6ec4e75b58a864e0b94bbd54637650dc887286726"
Sep 17 14:28:56.933379 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:56.933356    2582 scope.go:117] "RemoveContainer" containerID="015b660060515c2ef0e8448611ac637a732cd9c330dd99d69e03445a60aab678"
Sep 17 14:28:56.942259 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:56.942231    2582 scope.go:117] "RemoveContainer" containerID="924ff6cafa7b48480c3d85b6ec4e75b58a864e0b94bbd54637650dc887286726"
Sep 17 14:28:56.942574 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:28:56.942549    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"924ff6cafa7b48480c3d85b6ec4e75b58a864e0b94bbd54637650dc887286726\": container with ID starting with 924ff6cafa7b48480c3d85b6ec4e75b58a864e0b94bbd54637650dc887286726 not found: ID does not exist" containerID="924ff6cafa7b48480c3d85b6ec4e75b58a864e0b94bbd54637650dc887286726"
Sep 17 14:28:56.942689 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:56.942587    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"924ff6cafa7b48480c3d85b6ec4e75b58a864e0b94bbd54637650dc887286726"} err="failed to get container status \"924ff6cafa7b48480c3d85b6ec4e75b58a864e0b94bbd54637650dc887286726\": rpc error: code = NotFound desc = could not find container \"924ff6cafa7b48480c3d85b6ec4e75b58a864e0b94bbd54637650dc887286726\": container with ID starting with 924ff6cafa7b48480c3d85b6ec4e75b58a864e0b94bbd54637650dc887286726 not found: ID does not exist"
Sep 17 14:28:56.942689 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:56.942628    2582 scope.go:117] "RemoveContainer" containerID="015b660060515c2ef0e8448611ac637a732cd9c330dd99d69e03445a60aab678"
Sep 17 14:28:56.942917 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:28:56.942903    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"015b660060515c2ef0e8448611ac637a732cd9c330dd99d69e03445a60aab678\": container with ID starting with 015b660060515c2ef0e8448611ac637a732cd9c330dd99d69e03445a60aab678 not found: ID does not exist" containerID="015b660060515c2ef0e8448611ac637a732cd9c330dd99d69e03445a60aab678"
Sep 17 14:28:56.942962 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:56.942923    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"015b660060515c2ef0e8448611ac637a732cd9c330dd99d69e03445a60aab678"} err="failed to get container status \"015b660060515c2ef0e8448611ac637a732cd9c330dd99d69e03445a60aab678\": rpc error: code = NotFound desc = could not find container \"015b660060515c2ef0e8448611ac637a732cd9c330dd99d69e03445a60aab678\": container with ID starting with 015b660060515c2ef0e8448611ac637a732cd9c330dd99d69e03445a60aab678 not found: ID does not exist"
Sep 17 14:28:56.960407 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:56.960374    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-workload-certs\") pod \"3ab0e2f8-287d-4ba7-b06c-e3bef8457239\" (UID: \"3ab0e2f8-287d-4ba7-b06c-e3bef8457239\") "
Sep 17 14:28:56.960537 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:56.960419    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c8q5w\" (UniqueName: \"kubernetes.io/projected/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-kube-api-access-c8q5w\") pod \"3ab0e2f8-287d-4ba7-b06c-e3bef8457239\" (UID: \"3ab0e2f8-287d-4ba7-b06c-e3bef8457239\") "
Sep 17 14:28:56.960537 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:56.960450    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-istiod-ca-cert\") pod \"3ab0e2f8-287d-4ba7-b06c-e3bef8457239\" (UID: \"3ab0e2f8-287d-4ba7-b06c-e3bef8457239\") "
Sep 17 14:28:56.960537 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:56.960482    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-credential-socket\") pod \"3ab0e2f8-287d-4ba7-b06c-e3bef8457239\" (UID: \"3ab0e2f8-287d-4ba7-b06c-e3bef8457239\") "
Sep 17 14:28:56.960706 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:56.960569    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-istio-envoy\") pod \"3ab0e2f8-287d-4ba7-b06c-e3bef8457239\" (UID: \"3ab0e2f8-287d-4ba7-b06c-e3bef8457239\") "
Sep 17 14:28:56.960706 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:56.960599    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-istio-podinfo\") pod \"3ab0e2f8-287d-4ba7-b06c-e3bef8457239\" (UID: \"3ab0e2f8-287d-4ba7-b06c-e3bef8457239\") "
Sep 17 14:28:56.960706 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:56.960653    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-workload-socket\") pod \"3ab0e2f8-287d-4ba7-b06c-e3bef8457239\" (UID: \"3ab0e2f8-287d-4ba7-b06c-e3bef8457239\") "
Sep 17 14:28:56.960867 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:56.960707    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-istio-data\") pod \"3ab0e2f8-287d-4ba7-b06c-e3bef8457239\" (UID: \"3ab0e2f8-287d-4ba7-b06c-e3bef8457239\") "
Sep 17 14:28:56.960867 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:56.960743    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-istio-token\") pod \"3ab0e2f8-287d-4ba7-b06c-e3bef8457239\" (UID: \"3ab0e2f8-287d-4ba7-b06c-e3bef8457239\") "
Sep 17 14:28:56.960867 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:56.960747    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3ab0e2f8-287d-4ba7-b06c-e3bef8457239" (UID: "3ab0e2f8-287d-4ba7-b06c-e3bef8457239"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:28:56.961027 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:56.960856    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3ab0e2f8-287d-4ba7-b06c-e3bef8457239" (UID: "3ab0e2f8-287d-4ba7-b06c-e3bef8457239"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:28:56.961027 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:56.960937    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3ab0e2f8-287d-4ba7-b06c-e3bef8457239" (UID: "3ab0e2f8-287d-4ba7-b06c-e3bef8457239"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:28:56.961027 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:56.960974    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3ab0e2f8-287d-4ba7-b06c-e3bef8457239" (UID: "3ab0e2f8-287d-4ba7-b06c-e3bef8457239"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:28:56.961180 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:56.961085    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-istio-data" (OuterVolumeSpecName: "istio-data") pod "3ab0e2f8-287d-4ba7-b06c-e3bef8457239" (UID: "3ab0e2f8-287d-4ba7-b06c-e3bef8457239"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:28:56.961270 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:56.961246    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-workload-certs\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:28:56.961445 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:56.961425    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-istiod-ca-cert\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:28:56.961533 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:56.961447    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-credential-socket\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:28:56.961533 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:56.961462    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-workload-socket\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:28:56.961533 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:56.961477    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-istio-data\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:28:56.962904 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:56.962877    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-kube-api-access-c8q5w" (OuterVolumeSpecName: "kube-api-access-c8q5w") pod "3ab0e2f8-287d-4ba7-b06c-e3bef8457239" (UID: "3ab0e2f8-287d-4ba7-b06c-e3bef8457239"). InnerVolumeSpecName "kube-api-access-c8q5w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:28:56.962994 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:56.962965    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3ab0e2f8-287d-4ba7-b06c-e3bef8457239" (UID: "3ab0e2f8-287d-4ba7-b06c-e3bef8457239"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 14:28:56.963036 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:56.963012    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3ab0e2f8-287d-4ba7-b06c-e3bef8457239" (UID: "3ab0e2f8-287d-4ba7-b06c-e3bef8457239"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:28:56.963090 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:56.963074    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-istio-token" (OuterVolumeSpecName: "istio-token") pod "3ab0e2f8-287d-4ba7-b06c-e3bef8457239" (UID: "3ab0e2f8-287d-4ba7-b06c-e3bef8457239"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:28:57.062661 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:57.062586    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c8q5w\" (UniqueName: \"kubernetes.io/projected/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-kube-api-access-c8q5w\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:28:57.062661 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:57.062655    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-istio-envoy\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:28:57.062661 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:57.062665    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-istio-podinfo\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:28:57.062661 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:57.062674    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ab0e2f8-287d-4ba7-b06c-e3bef8457239-istio-token\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:28:57.249433 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:57.249391    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn"]
Sep 17 14:28:57.253378 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:57.253348    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-1a095586-openshift-default-df7756fbd-d2frn"]
Sep 17 14:28:57.494372 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:28:57.494289    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ab0e2f8-287d-4ba7-b06c-e3bef8457239" path="/var/lib/kubelet/pods/3ab0e2f8-287d-4ba7-b06c-e3bef8457239/volumes"
Sep 17 14:29:01.503718 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:01.503688    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:29:01.637850 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:01.637777    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:29:01.657341 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:01.657303    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46" podUID="f20fa859-9ed0-4465-ab5c-37adbcbf12a9" containerName="main" containerID="cri-o://c196079f64c3ca0c9604f90269fc3117618d7875000bedf5e134129658feff42" gracePeriod=45
Sep 17 14:29:01.926263 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:01.926235    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46"
Sep 17 14:29:01.960679 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:01.960593    2582 generic.go:356] "Generic (PLEG): container finished" podID="f20fa859-9ed0-4465-ab5c-37adbcbf12a9" containerID="c196079f64c3ca0c9604f90269fc3117618d7875000bedf5e134129658feff42" exitCode=0
Sep 17 14:29:01.960679 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:01.960664    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46"
Sep 17 14:29:01.960873 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:01.960684    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46" event={"ID":"f20fa859-9ed0-4465-ab5c-37adbcbf12a9","Type":"ContainerDied","Data":"c196079f64c3ca0c9604f90269fc3117618d7875000bedf5e134129658feff42"}
Sep 17 14:29:01.960873 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:01.960743    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46" event={"ID":"f20fa859-9ed0-4465-ab5c-37adbcbf12a9","Type":"ContainerDied","Data":"ba07637042f10f5c13d0d71430251677787b04042d7fd8c3e850d44abe405fc2"}
Sep 17 14:29:01.960873 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:01.960769    2582 scope.go:117] "RemoveContainer" containerID="c196079f64c3ca0c9604f90269fc3117618d7875000bedf5e134129658feff42"
Sep 17 14:29:01.972405 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:01.972383    2582 scope.go:117] "RemoveContainer" containerID="c196079f64c3ca0c9604f90269fc3117618d7875000bedf5e134129658feff42"
Sep 17 14:29:01.972824 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:29:01.972791    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c196079f64c3ca0c9604f90269fc3117618d7875000bedf5e134129658feff42\": container with ID starting with c196079f64c3ca0c9604f90269fc3117618d7875000bedf5e134129658feff42 not found: ID does not exist" containerID="c196079f64c3ca0c9604f90269fc3117618d7875000bedf5e134129658feff42"
Sep 17 14:29:01.972948 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:01.972837    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c196079f64c3ca0c9604f90269fc3117618d7875000bedf5e134129658feff42"} err="failed to get container status \"c196079f64c3ca0c9604f90269fc3117618d7875000bedf5e134129658feff42\": rpc error: code = NotFound desc = could not find container \"c196079f64c3ca0c9604f90269fc3117618d7875000bedf5e134129658feff42\": container with ID starting with c196079f64c3ca0c9604f90269fc3117618d7875000bedf5e134129658feff42 not found: ID does not exist"
Sep 17 14:29:02.015957 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.015920    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f20fa859-9ed0-4465-ab5c-37adbcbf12a9-model-cache\") pod \"f20fa859-9ed0-4465-ab5c-37adbcbf12a9\" (UID: \"f20fa859-9ed0-4465-ab5c-37adbcbf12a9\") "
Sep 17 14:29:02.016128 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.015969    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f20fa859-9ed0-4465-ab5c-37adbcbf12a9-home\") pod \"f20fa859-9ed0-4465-ab5c-37adbcbf12a9\" (UID: \"f20fa859-9ed0-4465-ab5c-37adbcbf12a9\") "
Sep 17 14:29:02.016128 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.016108    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f20fa859-9ed0-4465-ab5c-37adbcbf12a9-tmp-dir\") pod \"f20fa859-9ed0-4465-ab5c-37adbcbf12a9\" (UID: \"f20fa859-9ed0-4465-ab5c-37adbcbf12a9\") "
Sep 17 14:29:02.016217 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.016142    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f20fa859-9ed0-4465-ab5c-37adbcbf12a9-tls-certs\") pod \"f20fa859-9ed0-4465-ab5c-37adbcbf12a9\" (UID: \"f20fa859-9ed0-4465-ab5c-37adbcbf12a9\") "
Sep 17 14:29:02.016217 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.016208    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f20fa859-9ed0-4465-ab5c-37adbcbf12a9-dshm\") pod \"f20fa859-9ed0-4465-ab5c-37adbcbf12a9\" (UID: \"f20fa859-9ed0-4465-ab5c-37adbcbf12a9\") "
Sep 17 14:29:02.016327 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.016249    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x7s69\" (UniqueName: \"kubernetes.io/projected/f20fa859-9ed0-4465-ab5c-37adbcbf12a9-kube-api-access-x7s69\") pod \"f20fa859-9ed0-4465-ab5c-37adbcbf12a9\" (UID: \"f20fa859-9ed0-4465-ab5c-37adbcbf12a9\") "
Sep 17 14:29:02.016327 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.016314    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f20fa859-9ed0-4465-ab5c-37adbcbf12a9-home" (OuterVolumeSpecName: "home") pod "f20fa859-9ed0-4465-ab5c-37adbcbf12a9" (UID: "f20fa859-9ed0-4465-ab5c-37adbcbf12a9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:29:02.016413 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.016284    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f20fa859-9ed0-4465-ab5c-37adbcbf12a9-model-cache" (OuterVolumeSpecName: "model-cache") pod "f20fa859-9ed0-4465-ab5c-37adbcbf12a9" (UID: "f20fa859-9ed0-4465-ab5c-37adbcbf12a9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:29:02.016464 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.016434    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f20fa859-9ed0-4465-ab5c-37adbcbf12a9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f20fa859-9ed0-4465-ab5c-37adbcbf12a9" (UID: "f20fa859-9ed0-4465-ab5c-37adbcbf12a9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:29:02.016697 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.016672    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f20fa859-9ed0-4465-ab5c-37adbcbf12a9-model-cache\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:29:02.016697 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.016697    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f20fa859-9ed0-4465-ab5c-37adbcbf12a9-home\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:29:02.016853 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.016731    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f20fa859-9ed0-4465-ab5c-37adbcbf12a9-tmp-dir\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:29:02.018546 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.018517    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f20fa859-9ed0-4465-ab5c-37adbcbf12a9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f20fa859-9ed0-4465-ab5c-37adbcbf12a9" (UID: "f20fa859-9ed0-4465-ab5c-37adbcbf12a9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:29:02.018546 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.018527    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f20fa859-9ed0-4465-ab5c-37adbcbf12a9-dshm" (OuterVolumeSpecName: "dshm") pod "f20fa859-9ed0-4465-ab5c-37adbcbf12a9" (UID: "f20fa859-9ed0-4465-ab5c-37adbcbf12a9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:29:02.018704 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.018588    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f20fa859-9ed0-4465-ab5c-37adbcbf12a9-kube-api-access-x7s69" (OuterVolumeSpecName: "kube-api-access-x7s69") pod "f20fa859-9ed0-4465-ab5c-37adbcbf12a9" (UID: "f20fa859-9ed0-4465-ab5c-37adbcbf12a9"). InnerVolumeSpecName "kube-api-access-x7s69". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:29:02.118281 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.118230    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f20fa859-9ed0-4465-ab5c-37adbcbf12a9-dshm\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:29:02.118281 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.118274    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7s69\" (UniqueName: \"kubernetes.io/projected/f20fa859-9ed0-4465-ab5c-37adbcbf12a9-kube-api-access-x7s69\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:29:02.118281 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.118287    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f20fa859-9ed0-4465-ab5c-37adbcbf12a9-tls-certs\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:29:02.142234 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.142203    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:29:02.190280 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.190240    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-fcd9f3/distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn"]
Sep 17 14:29:02.190739 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.190721    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ab0e2f8-287d-4ba7-b06c-e3bef8457239" containerName="istio-proxy"
Sep 17 14:29:02.190739 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.190737    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ab0e2f8-287d-4ba7-b06c-e3bef8457239" containerName="istio-proxy"
Sep 17 14:29:02.190905 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.190744    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ab0e2f8-287d-4ba7-b06c-e3bef8457239" containerName="istio-proxy"
Sep 17 14:29:02.190905 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.190750    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ab0e2f8-287d-4ba7-b06c-e3bef8457239" containerName="istio-proxy"
Sep 17 14:29:02.190905 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.190763    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f20fa859-9ed0-4465-ab5c-37adbcbf12a9" containerName="main"
Sep 17 14:29:02.190905 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.190769    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f20fa859-9ed0-4465-ab5c-37adbcbf12a9" containerName="main"
Sep 17 14:29:02.190905 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.190783    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b259eb30-d5e7-4fde-b353-9dbcac62ece9" containerName="main"
Sep 17 14:29:02.190905 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.190788    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b259eb30-d5e7-4fde-b353-9dbcac62ece9" containerName="main"
Sep 17 14:29:02.190905 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.190860    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ab0e2f8-287d-4ba7-b06c-e3bef8457239" containerName="istio-proxy"
Sep 17 14:29:02.190905 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.190869    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f20fa859-9ed0-4465-ab5c-37adbcbf12a9" containerName="main"
Sep 17 14:29:02.190905 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.190876    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b259eb30-d5e7-4fde-b353-9dbcac62ece9" containerName="main"
Sep 17 14:29:02.193923 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.193904    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-fcd9f3/distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn"
Sep 17 14:29:02.196969 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.196947    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-fcd9f3\"/\"distinct-w4-fcd9f3-kserve-self-signed-certs\""
Sep 17 14:29:02.218281 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.218250    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-fcd9f3/distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn"]
Sep 17 14:29:02.267083 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.267047    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:29:02.284802 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.284717    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46"]
Sep 17 14:29:02.287147 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.287124    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-fcd9f3/e2e-conflict-a-9497140e-kserve-5d74d887fb-xtd46"]
Sep 17 14:29:02.319564 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.319531    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/775a4af7-36fe-429a-9fde-d85bc8ce5cfb-tmp-dir\") pod \"distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn\" (UID: \"775a4af7-36fe-429a-9fde-d85bc8ce5cfb\") " pod="e2e-models-e2e-worker-w4-fcd9f3/distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn"
Sep 17 14:29:02.319717 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.319570    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/775a4af7-36fe-429a-9fde-d85bc8ce5cfb-dshm\") pod \"distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn\" (UID: \"775a4af7-36fe-429a-9fde-d85bc8ce5cfb\") " pod="e2e-models-e2e-worker-w4-fcd9f3/distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn"
Sep 17 14:29:02.319777 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.319713    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/775a4af7-36fe-429a-9fde-d85bc8ce5cfb-tls-certs\") pod \"distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn\" (UID: \"775a4af7-36fe-429a-9fde-d85bc8ce5cfb\") " pod="e2e-models-e2e-worker-w4-fcd9f3/distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn"
Sep 17 14:29:02.319812 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.319775    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/775a4af7-36fe-429a-9fde-d85bc8ce5cfb-home\") pod \"distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn\" (UID: \"775a4af7-36fe-429a-9fde-d85bc8ce5cfb\") " pod="e2e-models-e2e-worker-w4-fcd9f3/distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn"
Sep 17 14:29:02.319855 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.319825    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-km5jb\" (UniqueName: \"kubernetes.io/projected/775a4af7-36fe-429a-9fde-d85bc8ce5cfb-kube-api-access-km5jb\") pod \"distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn\" (UID: \"775a4af7-36fe-429a-9fde-d85bc8ce5cfb\") " pod="e2e-models-e2e-worker-w4-fcd9f3/distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn"
Sep 17 14:29:02.319891 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.319856    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/775a4af7-36fe-429a-9fde-d85bc8ce5cfb-model-cache\") pod \"distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn\" (UID: \"775a4af7-36fe-429a-9fde-d85bc8ce5cfb\") " pod="e2e-models-e2e-worker-w4-fcd9f3/distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn"
Sep 17 14:29:02.421176 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.421138    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/775a4af7-36fe-429a-9fde-d85bc8ce5cfb-tmp-dir\") pod \"distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn\" (UID: \"775a4af7-36fe-429a-9fde-d85bc8ce5cfb\") " pod="e2e-models-e2e-worker-w4-fcd9f3/distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn"
Sep 17 14:29:02.421176 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.421176    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/775a4af7-36fe-429a-9fde-d85bc8ce5cfb-dshm\") pod \"distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn\" (UID: \"775a4af7-36fe-429a-9fde-d85bc8ce5cfb\") " pod="e2e-models-e2e-worker-w4-fcd9f3/distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn"
Sep 17 14:29:02.421422 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.421237    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/775a4af7-36fe-429a-9fde-d85bc8ce5cfb-tls-certs\") pod \"distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn\" (UID: \"775a4af7-36fe-429a-9fde-d85bc8ce5cfb\") " pod="e2e-models-e2e-worker-w4-fcd9f3/distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn"
Sep 17 14:29:02.421422 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.421280    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/775a4af7-36fe-429a-9fde-d85bc8ce5cfb-home\") pod \"distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn\" (UID: \"775a4af7-36fe-429a-9fde-d85bc8ce5cfb\") " pod="e2e-models-e2e-worker-w4-fcd9f3/distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn"
Sep 17 14:29:02.421422 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.421333    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-km5jb\" (UniqueName: \"kubernetes.io/projected/775a4af7-36fe-429a-9fde-d85bc8ce5cfb-kube-api-access-km5jb\") pod \"distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn\" (UID: \"775a4af7-36fe-429a-9fde-d85bc8ce5cfb\") " pod="e2e-models-e2e-worker-w4-fcd9f3/distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn"
Sep 17 14:29:02.421422 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.421375    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/775a4af7-36fe-429a-9fde-d85bc8ce5cfb-model-cache\") pod \"distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn\" (UID: \"775a4af7-36fe-429a-9fde-d85bc8ce5cfb\") " pod="e2e-models-e2e-worker-w4-fcd9f3/distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn"
Sep 17 14:29:02.421653 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.421530    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/775a4af7-36fe-429a-9fde-d85bc8ce5cfb-tmp-dir\") pod \"distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn\" (UID: \"775a4af7-36fe-429a-9fde-d85bc8ce5cfb\") " pod="e2e-models-e2e-worker-w4-fcd9f3/distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn"
Sep 17 14:29:02.421743 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.421723    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/775a4af7-36fe-429a-9fde-d85bc8ce5cfb-home\") pod \"distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn\" (UID: \"775a4af7-36fe-429a-9fde-d85bc8ce5cfb\") " pod="e2e-models-e2e-worker-w4-fcd9f3/distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn"
Sep 17 14:29:02.421803 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.421781    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/775a4af7-36fe-429a-9fde-d85bc8ce5cfb-model-cache\") pod \"distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn\" (UID: \"775a4af7-36fe-429a-9fde-d85bc8ce5cfb\") " pod="e2e-models-e2e-worker-w4-fcd9f3/distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn"
Sep 17 14:29:02.423553 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.423530    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/775a4af7-36fe-429a-9fde-d85bc8ce5cfb-dshm\") pod \"distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn\" (UID: \"775a4af7-36fe-429a-9fde-d85bc8ce5cfb\") " pod="e2e-models-e2e-worker-w4-fcd9f3/distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn"
Sep 17 14:29:02.423830 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.423815    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/775a4af7-36fe-429a-9fde-d85bc8ce5cfb-tls-certs\") pod \"distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn\" (UID: \"775a4af7-36fe-429a-9fde-d85bc8ce5cfb\") " pod="e2e-models-e2e-worker-w4-fcd9f3/distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn"
Sep 17 14:29:02.429598 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.429568    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-km5jb\" (UniqueName: \"kubernetes.io/projected/775a4af7-36fe-429a-9fde-d85bc8ce5cfb-kube-api-access-km5jb\") pod \"distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn\" (UID: \"775a4af7-36fe-429a-9fde-d85bc8ce5cfb\") " pod="e2e-models-e2e-worker-w4-fcd9f3/distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn"
Sep 17 14:29:02.503691 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.503653    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-fcd9f3/distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn"
Sep 17 14:29:02.809423 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.809358    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-fcd9f3/distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn"]
Sep 17 14:29:02.817009 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:29:02.816823    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod775a4af7_36fe_429a_9fde_d85bc8ce5cfb.slice/crio-a78762c0009b73a18e8b2ba897e49ddc1feb6ef160941346551699c8f2475a45 WatchSource:0}: Error finding container a78762c0009b73a18e8b2ba897e49ddc1feb6ef160941346551699c8f2475a45: Status 404 returned error can't find the container with id a78762c0009b73a18e8b2ba897e49ddc1feb6ef160941346551699c8f2475a45
Sep 17 14:29:02.971973 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.971884    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-fcd9f3/distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn" event={"ID":"775a4af7-36fe-429a-9fde-d85bc8ce5cfb","Type":"ContainerStarted","Data":"c50f45b7f62bea17926765fbbeb75eb036fc32d8325a2f72cf8242ab150b1c77"}
Sep 17 14:29:02.971973 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.971942    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-fcd9f3/distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn" event={"ID":"775a4af7-36fe-429a-9fde-d85bc8ce5cfb","Type":"ContainerStarted","Data":"a78762c0009b73a18e8b2ba897e49ddc1feb6ef160941346551699c8f2475a45"}
Sep 17 14:29:02.991128 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:02.991071    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-fcd9f3/distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn" podStartSLOduration=0.991051089 podStartE2EDuration="991.051089ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:29:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:29:02.989500146 +0000 UTC m=+1443.991043622" watchObservedRunningTime="2026-09-17 14:29:02.991051089 +0000 UTC m=+1443.992594564"
Sep 17 14:29:03.497138 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:03.497100    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f20fa859-9ed0-4465-ab5c-37adbcbf12a9" path="/var/lib/kubelet/pods/f20fa859-9ed0-4465-ab5c-37adbcbf12a9/volumes"
Sep 17 14:29:08.287689 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:08.287651    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:29:08.419672 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:08.419631    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:29:08.520225 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:08.520191    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:29:12.503881 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:12.503842    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-fcd9f3/distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn"
Sep 17 14:29:12.504296 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:12.503894    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-fcd9f3/distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn"
Sep 17 14:29:12.518712 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:12.518683    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-fcd9f3/distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn"
Sep 17 14:29:13.055409 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:13.055378    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-fcd9f3/distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn"
Sep 17 14:29:15.593054 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:15.593018    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-4c29a2/auth-test-model-4c29a2-kserve-8c7d98b47-6tn75"]
Sep 17 14:29:15.593902 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:15.593879    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ab0e2f8-287d-4ba7-b06c-e3bef8457239" containerName="istio-proxy"
Sep 17 14:29:15.597136 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:15.597117    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-4c29a2/auth-test-model-4c29a2-kserve-8c7d98b47-6tn75"
Sep 17 14:29:15.599719 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:15.599697    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-4c29a2\"/\"kube-root-ca.crt\""
Sep 17 14:29:15.600507 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:15.600485    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-4c29a2\"/\"openshift-service-ca.crt\""
Sep 17 14:29:15.600646 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:15.600581    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-4c29a2\"/\"auth-test-model-4c29a2-kserve-self-signed-certs\""
Sep 17 14:29:15.600646 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:15.600629    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-4c29a2\"/\"default-dockercfg-svpxf\""
Sep 17 14:29:15.605936 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:15.605876    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-4c29a2/auth-test-model-4c29a2-kserve-8c7d98b47-6tn75"]
Sep 17 14:29:15.632118 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:15.632085    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:29:15.672219 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:15.672189    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3731cd4a-3fe7-4a2c-8d28-bda2472f1042-tmp-dir\") pod \"auth-test-model-4c29a2-kserve-8c7d98b47-6tn75\" (UID: \"3731cd4a-3fe7-4a2c-8d28-bda2472f1042\") " pod="ai-tenant-e2e-shared-b-w3-4c29a2/auth-test-model-4c29a2-kserve-8c7d98b47-6tn75"
Sep 17 14:29:15.672372 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:15.672230    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3731cd4a-3fe7-4a2c-8d28-bda2472f1042-model-cache\") pod \"auth-test-model-4c29a2-kserve-8c7d98b47-6tn75\" (UID: \"3731cd4a-3fe7-4a2c-8d28-bda2472f1042\") " pod="ai-tenant-e2e-shared-b-w3-4c29a2/auth-test-model-4c29a2-kserve-8c7d98b47-6tn75"
Sep 17 14:29:15.672372 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:15.672345    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3731cd4a-3fe7-4a2c-8d28-bda2472f1042-home\") pod \"auth-test-model-4c29a2-kserve-8c7d98b47-6tn75\" (UID: \"3731cd4a-3fe7-4a2c-8d28-bda2472f1042\") " pod="ai-tenant-e2e-shared-b-w3-4c29a2/auth-test-model-4c29a2-kserve-8c7d98b47-6tn75"
Sep 17 14:29:15.672466 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:15.672384    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3731cd4a-3fe7-4a2c-8d28-bda2472f1042-tls-certs\") pod \"auth-test-model-4c29a2-kserve-8c7d98b47-6tn75\" (UID: \"3731cd4a-3fe7-4a2c-8d28-bda2472f1042\") " pod="ai-tenant-e2e-shared-b-w3-4c29a2/auth-test-model-4c29a2-kserve-8c7d98b47-6tn75"
Sep 17 14:29:15.672466 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:15.672458    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pq7c\" (UniqueName: \"kubernetes.io/projected/3731cd4a-3fe7-4a2c-8d28-bda2472f1042-kube-api-access-7pq7c\") pod \"auth-test-model-4c29a2-kserve-8c7d98b47-6tn75\" (UID: \"3731cd4a-3fe7-4a2c-8d28-bda2472f1042\") " pod="ai-tenant-e2e-shared-b-w3-4c29a2/auth-test-model-4c29a2-kserve-8c7d98b47-6tn75"
Sep 17 14:29:15.672546 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:15.672480    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3731cd4a-3fe7-4a2c-8d28-bda2472f1042-dshm\") pod \"auth-test-model-4c29a2-kserve-8c7d98b47-6tn75\" (UID: \"3731cd4a-3fe7-4a2c-8d28-bda2472f1042\") " pod="ai-tenant-e2e-shared-b-w3-4c29a2/auth-test-model-4c29a2-kserve-8c7d98b47-6tn75"
Sep 17 14:29:15.773937 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:15.773903    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pq7c\" (UniqueName: \"kubernetes.io/projected/3731cd4a-3fe7-4a2c-8d28-bda2472f1042-kube-api-access-7pq7c\") pod \"auth-test-model-4c29a2-kserve-8c7d98b47-6tn75\" (UID: \"3731cd4a-3fe7-4a2c-8d28-bda2472f1042\") " pod="ai-tenant-e2e-shared-b-w3-4c29a2/auth-test-model-4c29a2-kserve-8c7d98b47-6tn75"
Sep 17 14:29:15.774113 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:15.773946    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3731cd4a-3fe7-4a2c-8d28-bda2472f1042-dshm\") pod \"auth-test-model-4c29a2-kserve-8c7d98b47-6tn75\" (UID: \"3731cd4a-3fe7-4a2c-8d28-bda2472f1042\") " pod="ai-tenant-e2e-shared-b-w3-4c29a2/auth-test-model-4c29a2-kserve-8c7d98b47-6tn75"
Sep 17 14:29:15.774113 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:15.773976    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3731cd4a-3fe7-4a2c-8d28-bda2472f1042-tmp-dir\") pod \"auth-test-model-4c29a2-kserve-8c7d98b47-6tn75\" (UID: \"3731cd4a-3fe7-4a2c-8d28-bda2472f1042\") " pod="ai-tenant-e2e-shared-b-w3-4c29a2/auth-test-model-4c29a2-kserve-8c7d98b47-6tn75"
Sep 17 14:29:15.774113 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:15.774007    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3731cd4a-3fe7-4a2c-8d28-bda2472f1042-model-cache\") pod \"auth-test-model-4c29a2-kserve-8c7d98b47-6tn75\" (UID: \"3731cd4a-3fe7-4a2c-8d28-bda2472f1042\") " pod="ai-tenant-e2e-shared-b-w3-4c29a2/auth-test-model-4c29a2-kserve-8c7d98b47-6tn75"
Sep 17 14:29:15.774113 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:15.774069    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3731cd4a-3fe7-4a2c-8d28-bda2472f1042-home\") pod \"auth-test-model-4c29a2-kserve-8c7d98b47-6tn75\" (UID: \"3731cd4a-3fe7-4a2c-8d28-bda2472f1042\") " pod="ai-tenant-e2e-shared-b-w3-4c29a2/auth-test-model-4c29a2-kserve-8c7d98b47-6tn75"
Sep 17 14:29:15.774113 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:15.774093    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3731cd4a-3fe7-4a2c-8d28-bda2472f1042-tls-certs\") pod \"auth-test-model-4c29a2-kserve-8c7d98b47-6tn75\" (UID: \"3731cd4a-3fe7-4a2c-8d28-bda2472f1042\") " pod="ai-tenant-e2e-shared-b-w3-4c29a2/auth-test-model-4c29a2-kserve-8c7d98b47-6tn75"
Sep 17 14:29:15.774424 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:15.774402    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3731cd4a-3fe7-4a2c-8d28-bda2472f1042-tmp-dir\") pod \"auth-test-model-4c29a2-kserve-8c7d98b47-6tn75\" (UID: \"3731cd4a-3fe7-4a2c-8d28-bda2472f1042\") " pod="ai-tenant-e2e-shared-b-w3-4c29a2/auth-test-model-4c29a2-kserve-8c7d98b47-6tn75"
Sep 17 14:29:15.774521 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:15.774489    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3731cd4a-3fe7-4a2c-8d28-bda2472f1042-model-cache\") pod \"auth-test-model-4c29a2-kserve-8c7d98b47-6tn75\" (UID: \"3731cd4a-3fe7-4a2c-8d28-bda2472f1042\") " pod="ai-tenant-e2e-shared-b-w3-4c29a2/auth-test-model-4c29a2-kserve-8c7d98b47-6tn75"
Sep 17 14:29:15.774587 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:15.774557    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3731cd4a-3fe7-4a2c-8d28-bda2472f1042-home\") pod \"auth-test-model-4c29a2-kserve-8c7d98b47-6tn75\" (UID: \"3731cd4a-3fe7-4a2c-8d28-bda2472f1042\") " pod="ai-tenant-e2e-shared-b-w3-4c29a2/auth-test-model-4c29a2-kserve-8c7d98b47-6tn75"
Sep 17 14:29:15.776338 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:15.776321    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3731cd4a-3fe7-4a2c-8d28-bda2472f1042-dshm\") pod \"auth-test-model-4c29a2-kserve-8c7d98b47-6tn75\" (UID: \"3731cd4a-3fe7-4a2c-8d28-bda2472f1042\") " pod="ai-tenant-e2e-shared-b-w3-4c29a2/auth-test-model-4c29a2-kserve-8c7d98b47-6tn75"
Sep 17 14:29:15.776679 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:15.776628    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3731cd4a-3fe7-4a2c-8d28-bda2472f1042-tls-certs\") pod \"auth-test-model-4c29a2-kserve-8c7d98b47-6tn75\" (UID: \"3731cd4a-3fe7-4a2c-8d28-bda2472f1042\") " pod="ai-tenant-e2e-shared-b-w3-4c29a2/auth-test-model-4c29a2-kserve-8c7d98b47-6tn75"
Sep 17 14:29:15.782634 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:15.782592    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pq7c\" (UniqueName: \"kubernetes.io/projected/3731cd4a-3fe7-4a2c-8d28-bda2472f1042-kube-api-access-7pq7c\") pod \"auth-test-model-4c29a2-kserve-8c7d98b47-6tn75\" (UID: \"3731cd4a-3fe7-4a2c-8d28-bda2472f1042\") " pod="ai-tenant-e2e-shared-b-w3-4c29a2/auth-test-model-4c29a2-kserve-8c7d98b47-6tn75"
Sep 17 14:29:15.908576 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:15.908490    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-4c29a2/auth-test-model-4c29a2-kserve-8c7d98b47-6tn75"
Sep 17 14:29:16.196038 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:16.196011    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-4c29a2/auth-test-model-4c29a2-kserve-8c7d98b47-6tn75"]
Sep 17 14:29:16.197347 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:29:16.197323    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3731cd4a_3fe7_4a2c_8d28_bda2472f1042.slice/crio-60b169de097d54703f3e4fb03c5c92d246bc2df14e592cd97776dd60d2223c43 WatchSource:0}: Error finding container 60b169de097d54703f3e4fb03c5c92d246bc2df14e592cd97776dd60d2223c43: Status 404 returned error can't find the container with id 60b169de097d54703f3e4fb03c5c92d246bc2df14e592cd97776dd60d2223c43
Sep 17 14:29:17.069145 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:17.069111    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-4c29a2/auth-test-model-4c29a2-kserve-8c7d98b47-6tn75" event={"ID":"3731cd4a-3fe7-4a2c-8d28-bda2472f1042","Type":"ContainerStarted","Data":"5310629e2035b1102f6aef8fbdfbfac36d837eb9a00be5ba3d8e50a34f236d49"}
Sep 17 14:29:17.069524 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:17.069157    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-4c29a2/auth-test-model-4c29a2-kserve-8c7d98b47-6tn75" event={"ID":"3731cd4a-3fe7-4a2c-8d28-bda2472f1042","Type":"ContainerStarted","Data":"60b169de097d54703f3e4fb03c5c92d246bc2df14e592cd97776dd60d2223c43"}
Sep 17 14:29:17.093983 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:17.093935    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-4c29a2/auth-test-model-4c29a2-kserve-8c7d98b47-6tn75" podStartSLOduration=2.093919463 podStartE2EDuration="2.093919463s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:29:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:29:17.091381534 +0000 UTC m=+1458.092925007" watchObservedRunningTime="2026-09-17 14:29:17.093919463 +0000 UTC m=+1458.095462938"
Sep 17 14:29:18.994827 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:18.994784    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:29:19.226538 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:19.226504    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:29:20.699498 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:20.699464    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:29:23.324029 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:23.323983    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:29:25.909486 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:25.909449    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-4c29a2/auth-test-model-4c29a2-kserve-8c7d98b47-6tn75"
Sep 17 14:29:25.909885 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:25.909721    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-4c29a2/auth-test-model-4c29a2-kserve-8c7d98b47-6tn75"
Sep 17 14:29:25.922257 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:25.922233    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-4c29a2/auth-test-model-4c29a2-kserve-8c7d98b47-6tn75"
Sep 17 14:29:26.140734 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:26.140706    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-4c29a2/auth-test-model-4c29a2-kserve-8c7d98b47-6tn75"
Sep 17 14:29:38.430968 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:38.430924    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:29:39.642510 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:39.642479    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:29:40.259508 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:40.259475    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:29:46.077907 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:46.077863    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:29:46.201801 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:46.201762    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:29:46.422201 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:46.422096    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:29:46.663151 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:46.663117    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv"]
Sep 17 14:29:46.667592 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:46.667571    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv"
Sep 17 14:29:46.670136 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:46.670112    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-a954c8-openshift-default-dockercfg-9jmqx\""
Sep 17 14:29:46.680979 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:46.680956    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv"]
Sep 17 14:29:46.705925 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:46.705899    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7be60b88-b959-4272-a784-e6e96ab10098-istio-data\") pod \"e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv\" (UID: \"7be60b88-b959-4272-a784-e6e96ab10098\") " pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv"
Sep 17 14:29:46.706073 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:46.705941    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7be60b88-b959-4272-a784-e6e96ab10098-credential-socket\") pod \"e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv\" (UID: \"7be60b88-b959-4272-a784-e6e96ab10098\") " pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv"
Sep 17 14:29:46.706073 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:46.705970    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7be60b88-b959-4272-a784-e6e96ab10098-workload-certs\") pod \"e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv\" (UID: \"7be60b88-b959-4272-a784-e6e96ab10098\") " pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv"
Sep 17 14:29:46.706165 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:46.706096    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7be60b88-b959-4272-a784-e6e96ab10098-istiod-ca-cert\") pod \"e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv\" (UID: \"7be60b88-b959-4272-a784-e6e96ab10098\") " pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv"
Sep 17 14:29:46.706165 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:46.706132    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7be60b88-b959-4272-a784-e6e96ab10098-istio-token\") pod \"e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv\" (UID: \"7be60b88-b959-4272-a784-e6e96ab10098\") " pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv"
Sep 17 14:29:46.706235 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:46.706176    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dckps\" (UniqueName: \"kubernetes.io/projected/7be60b88-b959-4272-a784-e6e96ab10098-kube-api-access-dckps\") pod \"e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv\" (UID: \"7be60b88-b959-4272-a784-e6e96ab10098\") " pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv"
Sep 17 14:29:46.706235 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:46.706204    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7be60b88-b959-4272-a784-e6e96ab10098-istio-envoy\") pod \"e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv\" (UID: \"7be60b88-b959-4272-a784-e6e96ab10098\") " pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv"
Sep 17 14:29:46.706303 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:46.706262    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7be60b88-b959-4272-a784-e6e96ab10098-workload-socket\") pod \"e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv\" (UID: \"7be60b88-b959-4272-a784-e6e96ab10098\") " pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv"
Sep 17 14:29:46.706303 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:46.706276    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7be60b88-b959-4272-a784-e6e96ab10098-istio-podinfo\") pod \"e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv\" (UID: \"7be60b88-b959-4272-a784-e6e96ab10098\") " pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv"
Sep 17 14:29:46.784441 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:46.784390    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:29:46.806884 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:46.806849    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7be60b88-b959-4272-a784-e6e96ab10098-istiod-ca-cert\") pod \"e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv\" (UID: \"7be60b88-b959-4272-a784-e6e96ab10098\") " pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv"
Sep 17 14:29:46.807095 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:46.806897    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7be60b88-b959-4272-a784-e6e96ab10098-istio-token\") pod \"e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv\" (UID: \"7be60b88-b959-4272-a784-e6e96ab10098\") " pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv"
Sep 17 14:29:46.807095 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:46.806923    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dckps\" (UniqueName: \"kubernetes.io/projected/7be60b88-b959-4272-a784-e6e96ab10098-kube-api-access-dckps\") pod \"e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv\" (UID: \"7be60b88-b959-4272-a784-e6e96ab10098\") " pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv"
Sep 17 14:29:46.807095 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:46.806948    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7be60b88-b959-4272-a784-e6e96ab10098-istio-envoy\") pod \"e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv\" (UID: \"7be60b88-b959-4272-a784-e6e96ab10098\") " pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv"
Sep 17 14:29:46.807095 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:46.807001    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7be60b88-b959-4272-a784-e6e96ab10098-workload-socket\") pod \"e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv\" (UID: \"7be60b88-b959-4272-a784-e6e96ab10098\") " pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv"
Sep 17 14:29:46.807095 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:46.807022    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7be60b88-b959-4272-a784-e6e96ab10098-istio-podinfo\") pod \"e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv\" (UID: \"7be60b88-b959-4272-a784-e6e96ab10098\") " pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv"
Sep 17 14:29:46.807095 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:46.807050    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7be60b88-b959-4272-a784-e6e96ab10098-istio-data\") pod \"e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv\" (UID: \"7be60b88-b959-4272-a784-e6e96ab10098\") " pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv"
Sep 17 14:29:46.807095 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:46.807089    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7be60b88-b959-4272-a784-e6e96ab10098-credential-socket\") pod \"e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv\" (UID: \"7be60b88-b959-4272-a784-e6e96ab10098\") " pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv"
Sep 17 14:29:46.807482 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:46.807131    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7be60b88-b959-4272-a784-e6e96ab10098-workload-certs\") pod \"e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv\" (UID: \"7be60b88-b959-4272-a784-e6e96ab10098\") " pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv"
Sep 17 14:29:46.807482 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:46.807442    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7be60b88-b959-4272-a784-e6e96ab10098-workload-socket\") pod \"e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv\" (UID: \"7be60b88-b959-4272-a784-e6e96ab10098\") " pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv"
Sep 17 14:29:46.807590 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:46.807515    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7be60b88-b959-4272-a784-e6e96ab10098-istio-data\") pod \"e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv\" (UID: \"7be60b88-b959-4272-a784-e6e96ab10098\") " pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv"
Sep 17 14:29:46.807590 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:46.807580    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7be60b88-b959-4272-a784-e6e96ab10098-credential-socket\") pod \"e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv\" (UID: \"7be60b88-b959-4272-a784-e6e96ab10098\") " pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv"
Sep 17 14:29:46.807787 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:46.807598    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7be60b88-b959-4272-a784-e6e96ab10098-workload-certs\") pod \"e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv\" (UID: \"7be60b88-b959-4272-a784-e6e96ab10098\") " pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv"
Sep 17 14:29:46.807989 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:46.807968    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7be60b88-b959-4272-a784-e6e96ab10098-istiod-ca-cert\") pod \"e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv\" (UID: \"7be60b88-b959-4272-a784-e6e96ab10098\") " pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv"
Sep 17 14:29:46.809470 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:46.809450    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7be60b88-b959-4272-a784-e6e96ab10098-istio-envoy\") pod \"e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv\" (UID: \"7be60b88-b959-4272-a784-e6e96ab10098\") " pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv"
Sep 17 14:29:46.809810 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:46.809792    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7be60b88-b959-4272-a784-e6e96ab10098-istio-podinfo\") pod \"e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv\" (UID: \"7be60b88-b959-4272-a784-e6e96ab10098\") " pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv"
Sep 17 14:29:46.814707 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:46.814687    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7be60b88-b959-4272-a784-e6e96ab10098-istio-token\") pod \"e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv\" (UID: \"7be60b88-b959-4272-a784-e6e96ab10098\") " pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv"
Sep 17 14:29:46.815022 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:46.815003    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dckps\" (UniqueName: \"kubernetes.io/projected/7be60b88-b959-4272-a784-e6e96ab10098-kube-api-access-dckps\") pod \"e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv\" (UID: \"7be60b88-b959-4272-a784-e6e96ab10098\") " pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv"
Sep 17 14:29:46.979367 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:46.979332    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv"
Sep 17 14:29:47.100539 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:47.100503    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:29:47.262899 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:47.262822    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv"]
Sep 17 14:29:47.266661 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:29:47.266595    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7be60b88_b959_4272_a784_e6e96ab10098.slice/crio-77aa303a62c48ac5afbdb08ae3750ffc552fd8795bc7e05c8885ad699229bd52 WatchSource:0}: Error finding container 77aa303a62c48ac5afbdb08ae3750ffc552fd8795bc7e05c8885ad699229bd52: Status 404 returned error can't find the container with id 77aa303a62c48ac5afbdb08ae3750ffc552fd8795bc7e05c8885ad699229bd52
Sep 17 14:29:47.269552 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:47.269509    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:29:47.269675 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:47.269640    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:29:47.269726 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:47.269693    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:29:48.271489 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:48.271448    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv" event={"ID":"7be60b88-b959-4272-a784-e6e96ab10098","Type":"ContainerStarted","Data":"452eadf230bb93dd03220d6cc458c409dbc6952adc525b7d1569de2abad8805c"}
Sep 17 14:29:48.271875 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:48.271533    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv" event={"ID":"7be60b88-b959-4272-a784-e6e96ab10098","Type":"ContainerStarted","Data":"77aa303a62c48ac5afbdb08ae3750ffc552fd8795bc7e05c8885ad699229bd52"}
Sep 17 14:29:48.296655 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:48.296578    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv" podStartSLOduration=2.296563584 podStartE2EDuration="2.296563584s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:29:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:29:48.294860414 +0000 UTC m=+1489.296403888" watchObservedRunningTime="2026-09-17 14:29:48.296563584 +0000 UTC m=+1489.298107057"
Sep 17 14:29:48.982245 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:48.982202    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv"
Sep 17 14:29:49.016825 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:49.016793    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv"
Sep 17 14:29:49.285492 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:49.285394    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv"
Sep 17 14:29:49.287506 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:49.287177    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv"
Sep 17 14:29:49.555300 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:49.555190    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:29:49.738432 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:49.738390    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:29:50.295552 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:50.295522    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:29:50.399020 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:50.398986    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:29:50.687893 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:50.687800    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:29:51.158427 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:51.158393    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:29:51.909781 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:51.909749    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:29:52.766634 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:52.766564    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:29:53.033006 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:53.032884    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:29:53.547926 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:53.547889    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:29:54.352568 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:54.352533    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:29:55.044346 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:55.044314    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:29:55.142747 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:55.142708    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:29:55.379440 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:55.379346    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:29:56.831025 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:56.830994    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:29:56.921131 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:56.921101    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:29:59.571057 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:59.571025    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-rr97k_45ffa13d-635a-46e8-8894-30902b9cf719/console-operator/2.log"
Sep 17 14:29:59.574281 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:59.574252    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-rr97k_45ffa13d-635a-46e8-8894-30902b9cf719/console-operator/2.log"
Sep 17 14:29:59.735559 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:59.735525    2582 scope.go:117] "RemoveContainer" containerID="e5423e518fd285e9195d0b6f52909a9289a88c94f8cb0047542316cd72631f64"
Sep 17 14:29:59.744318 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:29:59.744294    2582 scope.go:117] "RemoveContainer" containerID="391e397e164a0fd2fb4805a77466f49c3ea4ab9eb459bd6ff53ee1589049ee50"
Sep 17 14:30:00.142693 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:00.142657    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-1269b2-29827590-rph8m"]
Sep 17 14:30:00.147090 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:00.147062    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-1269b2-29827590-rph8m"
Sep 17 14:30:00.148073 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:00.148049    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a954c8-29827590-r7j5l"]
Sep 17 14:30:00.150220 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:00.150196    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ddwrn\""
Sep 17 14:30:00.151565 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:00.151547    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-fcd9f3-29827590-dfxb7"]
Sep 17 14:30:00.151711 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:00.151689    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a954c8-29827590-r7j5l"
Sep 17 14:30:00.154968 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:00.154945    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-b83f70-29827590-knbmv"]
Sep 17 14:30:00.155109 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:00.155089    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-fcd9f3-29827590-dfxb7"
Sep 17 14:30:00.159104 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:00.159085    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-cc3801-29827590-96rhs"]
Sep 17 14:30:00.159302 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:00.159197    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-b83f70-29827590-knbmv"
Sep 17 14:30:00.160389 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:00.160365    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-1269b2-29827590-rph8m"
Sep 17 14:30:00.164234 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:00.164211    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-1269b2-29827590-rph8m"]
Sep 17 14:30:00.164398 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:00.164385    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a954c8-29827590-r7j5l"]
Sep 17 14:30:00.164597 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:00.164583    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-cc3801-29827590-96rhs"
Sep 17 14:30:00.170353 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:00.165588    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-fcd9f3-29827590-dfxb7"]
Sep 17 14:30:00.170353 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:00.169044    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-cc3801-29827590-96rhs"]
Sep 17 14:30:00.178233 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:00.174956    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a954c8-29827590-r7j5l"
Sep 17 14:30:00.180627 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:00.180583    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29827590-ncd6j"]
Sep 17 14:30:00.186601 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:00.186576    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29827590-ncd6j"
Sep 17 14:30:00.189357 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:00.189325    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-b83f70-29827590-knbmv"]
Sep 17 14:30:00.195253 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:00.194974    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29827590-ncd6j"]
Sep 17 14:30:00.235944 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:00.235903    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-fcd9f3-29827590-dfxb7"
Sep 17 14:30:00.250817 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:00.250389    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-b83f70-29827590-knbmv"
Sep 17 14:30:00.268124 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:00.266982    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-cc3801-29827590-96rhs"
Sep 17 14:30:00.314800 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:00.314750    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29827590-ncd6j"
Sep 17 14:30:00.464396 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:00.464368    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-1269b2-29827590-rph8m"]
Sep 17 14:30:00.467244 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:30:00.467217    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9a93782_7587_44b5_a57b_1e6522631a2a.slice/crio-8499d84e429a7c1feac66edd938315f529b74f3910a5aec93a68dee55832d29c WatchSource:0}: Error finding container 8499d84e429a7c1feac66edd938315f529b74f3910a5aec93a68dee55832d29c: Status 404 returned error can't find the container with id 8499d84e429a7c1feac66edd938315f529b74f3910a5aec93a68dee55832d29c
Sep 17 14:30:00.504018 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:00.503915    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a954c8-29827590-r7j5l"]
Sep 17 14:30:00.579691 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:00.579665    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-fcd9f3-29827590-dfxb7"]
Sep 17 14:30:00.581237 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:30:00.581204    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda92a7aa0_ad51_4c3c_a0b0_a25972496d3a.slice/crio-a5e3b4f61495adcfd9b64f4edc15243d47a031b62cbca45df510d279caafc604 WatchSource:0}: Error finding container a5e3b4f61495adcfd9b64f4edc15243d47a031b62cbca45df510d279caafc604: Status 404 returned error can't find the container with id a5e3b4f61495adcfd9b64f4edc15243d47a031b62cbca45df510d279caafc604
Sep 17 14:30:00.587662 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:00.587639    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-b83f70-29827590-knbmv"]
Sep 17 14:30:00.589179 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:30:00.589159    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a376b4d_87db_4a35_b4a7_d0e34ab90b47.slice/crio-538440f654e918a1372cc004f0f68d1e5292118370723b64f8e1c8480a1923c6 WatchSource:0}: Error finding container 538440f654e918a1372cc004f0f68d1e5292118370723b64f8e1c8480a1923c6: Status 404 returned error can't find the container with id 538440f654e918a1372cc004f0f68d1e5292118370723b64f8e1c8480a1923c6
Sep 17 14:30:00.597303 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:00.597277    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-cc3801-29827590-96rhs"]
Sep 17 14:30:00.598747 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:30:00.598723    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6d9f887_eea1_488c_abc4_01981a9e8724.slice/crio-1137427fceee72b29f509e517a7c6073cb7092521971e73d70f2622f03f5d7f4 WatchSource:0}: Error finding container 1137427fceee72b29f509e517a7c6073cb7092521971e73d70f2622f03f5d7f4: Status 404 returned error can't find the container with id 1137427fceee72b29f509e517a7c6073cb7092521971e73d70f2622f03f5d7f4
Sep 17 14:30:00.645190 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:00.645159    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29827590-ncd6j"]
Sep 17 14:30:00.733099 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:30:00.732944    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82b52f92_7f30_4db4_b1fc_b013003c0e13.slice/crio-26c80cd9197d836bd56cc7914a295ca0bc5addd0fc1c463011b645f0e455e781 WatchSource:0}: Error finding container 26c80cd9197d836bd56cc7914a295ca0bc5addd0fc1c463011b645f0e455e781: Status 404 returned error can't find the container with id 26c80cd9197d836bd56cc7914a295ca0bc5addd0fc1c463011b645f0e455e781
Sep 17 14:30:01.371907 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:01.371816    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-1269b2-29827590-rph8m" event={"ID":"c9a93782-7587-44b5-a57b-1e6522631a2a","Type":"ContainerStarted","Data":"00571ddd2965add6034aeff80c1ad6bc986807d101d08e6c5b325334b569609e"}
Sep 17 14:30:01.371907 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:01.371879    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-1269b2-29827590-rph8m" event={"ID":"c9a93782-7587-44b5-a57b-1e6522631a2a","Type":"ContainerStarted","Data":"8499d84e429a7c1feac66edd938315f529b74f3910a5aec93a68dee55832d29c"}
Sep 17 14:30:01.373425 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:01.373398    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a954c8-29827590-r7j5l" event={"ID":"1e111231-9395-400a-834f-e1e3a0215df3","Type":"ContainerStarted","Data":"d2c73eeac888155ca4c02534a08a4a4025b2dc937b39291e830c75cdf504e02c"}
Sep 17 14:30:01.373554 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:01.373448    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a954c8-29827590-r7j5l" event={"ID":"1e111231-9395-400a-834f-e1e3a0215df3","Type":"ContainerStarted","Data":"d4c4658983c179b98dbe14cd1a2678770f38397c548d94cd56bacd6330617192"}
Sep 17 14:30:01.374986 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:01.374964    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-cc3801-29827590-96rhs" event={"ID":"f6d9f887-eea1-488c-abc4-01981a9e8724","Type":"ContainerStarted","Data":"3e3f50fb650f05876336c9cac251291c20a3b6c268170fa5765899c2a6c7b17b"}
Sep 17 14:30:01.375095 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:01.375002    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-cc3801-29827590-96rhs" event={"ID":"f6d9f887-eea1-488c-abc4-01981a9e8724","Type":"ContainerStarted","Data":"1137427fceee72b29f509e517a7c6073cb7092521971e73d70f2622f03f5d7f4"}
Sep 17 14:30:01.376442 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:01.376421    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-b83f70-29827590-knbmv" event={"ID":"3a376b4d-87db-4a35-b4a7-d0e34ab90b47","Type":"ContainerStarted","Data":"feb6891c586219001cf26b49cdecc61f408b6537254c288b98413779bf397e8f"}
Sep 17 14:30:01.376535 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:01.376461    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-b83f70-29827590-knbmv" event={"ID":"3a376b4d-87db-4a35-b4a7-d0e34ab90b47","Type":"ContainerStarted","Data":"538440f654e918a1372cc004f0f68d1e5292118370723b64f8e1c8480a1923c6"}
Sep 17 14:30:01.377983 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:01.377960    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-fcd9f3-29827590-dfxb7" event={"ID":"a92a7aa0-ad51-4c3c-a0b0-a25972496d3a","Type":"ContainerStarted","Data":"a08616140d6acf8485d28366e2114b8095794d8bd32d5ff8280f06717da14ac2"}
Sep 17 14:30:01.378089 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:01.378003    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-fcd9f3-29827590-dfxb7" event={"ID":"a92a7aa0-ad51-4c3c-a0b0-a25972496d3a","Type":"ContainerStarted","Data":"a5e3b4f61495adcfd9b64f4edc15243d47a031b62cbca45df510d279caafc604"}
Sep 17 14:30:01.379335 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:01.379314    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29827590-ncd6j" event={"ID":"82b52f92-7f30-4db4-b1fc-b013003c0e13","Type":"ContainerStarted","Data":"a28176304b00506ead0258d1c58d61bfd3f44cd8e7feca9ae82789f250cbb969"}
Sep 17 14:30:01.379426 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:01.379349    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29827590-ncd6j" event={"ID":"82b52f92-7f30-4db4-b1fc-b013003c0e13","Type":"ContainerStarted","Data":"26c80cd9197d836bd56cc7914a295ca0bc5addd0fc1c463011b645f0e455e781"}
Sep 17 14:30:01.391585 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:01.391542    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-1269b2-29827590-rph8m" podStartSLOduration=1.391529131 podStartE2EDuration="1.391529131s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:30:01.388849856 +0000 UTC m=+1502.390393323" watchObservedRunningTime="2026-09-17 14:30:01.391529131 +0000 UTC m=+1502.393072604"
Sep 17 14:30:01.405499 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:01.405455    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29827590-ncd6j" podStartSLOduration=1.405442266 podStartE2EDuration="1.405442266s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:30:01.404732729 +0000 UTC m=+1502.406276195" watchObservedRunningTime="2026-09-17 14:30:01.405442266 +0000 UTC m=+1502.406985743"
Sep 17 14:30:01.421101 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:01.421040    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-cc3801-29827590-96rhs" podStartSLOduration=1.4210175010000001 podStartE2EDuration="1.421017501s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:30:01.420317777 +0000 UTC m=+1502.421861251" watchObservedRunningTime="2026-09-17 14:30:01.421017501 +0000 UTC m=+1502.422560975"
Sep 17 14:30:01.437258 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:01.437188    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-b83f70-29827590-knbmv" podStartSLOduration=1.4371676820000001 podStartE2EDuration="1.437167682s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:30:01.436392373 +0000 UTC m=+1502.437935851" watchObservedRunningTime="2026-09-17 14:30:01.437167682 +0000 UTC m=+1502.438711157"
Sep 17 14:30:01.451216 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:01.451172    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-fcd9f3-29827590-dfxb7" podStartSLOduration=1.4511562279999999 podStartE2EDuration="1.451156228s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:30:01.450216552 +0000 UTC m=+1502.451760027" watchObservedRunningTime="2026-09-17 14:30:01.451156228 +0000 UTC m=+1502.452699702"
Sep 17 14:30:01.468095 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:01.468049    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a954c8-29827590-r7j5l" podStartSLOduration=1.4680344330000001 podStartE2EDuration="1.468034433s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:30:01.465963496 +0000 UTC m=+1502.467506970" watchObservedRunningTime="2026-09-17 14:30:01.468034433 +0000 UTC m=+1502.469577957"
Sep 17 14:30:02.388011 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:02.387920    2582 generic.go:356] "Generic (PLEG): container finished" podID="3a376b4d-87db-4a35-b4a7-d0e34ab90b47" containerID="feb6891c586219001cf26b49cdecc61f408b6537254c288b98413779bf397e8f" exitCode=0
Sep 17 14:30:02.388011 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:02.387972    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-b83f70-29827590-knbmv" event={"ID":"3a376b4d-87db-4a35-b4a7-d0e34ab90b47","Type":"ContainerDied","Data":"feb6891c586219001cf26b49cdecc61f408b6537254c288b98413779bf397e8f"}
Sep 17 14:30:02.389595 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:02.389570    2582 generic.go:356] "Generic (PLEG): container finished" podID="a92a7aa0-ad51-4c3c-a0b0-a25972496d3a" containerID="a08616140d6acf8485d28366e2114b8095794d8bd32d5ff8280f06717da14ac2" exitCode=0
Sep 17 14:30:02.389787 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:02.389654    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-fcd9f3-29827590-dfxb7" event={"ID":"a92a7aa0-ad51-4c3c-a0b0-a25972496d3a","Type":"ContainerDied","Data":"a08616140d6acf8485d28366e2114b8095794d8bd32d5ff8280f06717da14ac2"}
Sep 17 14:30:02.391177 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:02.391151    2582 generic.go:356] "Generic (PLEG): container finished" podID="82b52f92-7f30-4db4-b1fc-b013003c0e13" containerID="a28176304b00506ead0258d1c58d61bfd3f44cd8e7feca9ae82789f250cbb969" exitCode=0
Sep 17 14:30:02.391302 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:02.391231    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29827590-ncd6j" event={"ID":"82b52f92-7f30-4db4-b1fc-b013003c0e13","Type":"ContainerDied","Data":"a28176304b00506ead0258d1c58d61bfd3f44cd8e7feca9ae82789f250cbb969"}
Sep 17 14:30:02.392791 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:02.392772    2582 generic.go:356] "Generic (PLEG): container finished" podID="c9a93782-7587-44b5-a57b-1e6522631a2a" containerID="00571ddd2965add6034aeff80c1ad6bc986807d101d08e6c5b325334b569609e" exitCode=0
Sep 17 14:30:02.392892 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:02.392836    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-1269b2-29827590-rph8m" event={"ID":"c9a93782-7587-44b5-a57b-1e6522631a2a","Type":"ContainerDied","Data":"00571ddd2965add6034aeff80c1ad6bc986807d101d08e6c5b325334b569609e"}
Sep 17 14:30:02.394274 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:02.394256    2582 generic.go:356] "Generic (PLEG): container finished" podID="1e111231-9395-400a-834f-e1e3a0215df3" containerID="d2c73eeac888155ca4c02534a08a4a4025b2dc937b39291e830c75cdf504e02c" exitCode=7
Sep 17 14:30:02.394375 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:02.394274    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a954c8-29827590-r7j5l" event={"ID":"1e111231-9395-400a-834f-e1e3a0215df3","Type":"ContainerDied","Data":"d2c73eeac888155ca4c02534a08a4a4025b2dc937b39291e830c75cdf504e02c"}
Sep 17 14:30:02.394492 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:02.394477    2582 scope.go:117] "RemoveContainer" containerID="d2c73eeac888155ca4c02534a08a4a4025b2dc937b39291e830c75cdf504e02c"
Sep 17 14:30:02.395895 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:02.395873    2582 generic.go:356] "Generic (PLEG): container finished" podID="f6d9f887-eea1-488c-abc4-01981a9e8724" containerID="3e3f50fb650f05876336c9cac251291c20a3b6c268170fa5765899c2a6c7b17b" exitCode=0
Sep 17 14:30:02.395985 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:02.395893    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-cc3801-29827590-96rhs" event={"ID":"f6d9f887-eea1-488c-abc4-01981a9e8724","Type":"ContainerDied","Data":"3e3f50fb650f05876336c9cac251291c20a3b6c268170fa5765899c2a6c7b17b"}
Sep 17 14:30:03.408095 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:03.408046    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a954c8-29827590-r7j5l" event={"ID":"1e111231-9395-400a-834f-e1e3a0215df3","Type":"ContainerStarted","Data":"29e47ec63ccc365eda39e8bbc6729b060249b6099a9c821190132119bfa5a67d"}
Sep 17 14:30:03.588446 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:03.588208    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-b83f70-29827590-knbmv"
Sep 17 14:30:03.631513 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:03.631235    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-1269b2-29827590-rph8m"
Sep 17 14:30:03.702023 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:03.701982    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29827590-ncd6j"
Sep 17 14:30:03.705184 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:03.705158    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-fcd9f3-29827590-dfxb7"
Sep 17 14:30:03.709025 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:03.708999    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-cc3801-29827590-96rhs"
Sep 17 14:30:04.415973 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:04.415940    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-b83f70-29827590-knbmv"
Sep 17 14:30:04.416438 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:04.415936    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-b83f70-29827590-knbmv" event={"ID":"3a376b4d-87db-4a35-b4a7-d0e34ab90b47","Type":"ContainerDied","Data":"538440f654e918a1372cc004f0f68d1e5292118370723b64f8e1c8480a1923c6"}
Sep 17 14:30:04.416438 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:04.416124    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="538440f654e918a1372cc004f0f68d1e5292118370723b64f8e1c8480a1923c6"
Sep 17 14:30:04.417426 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:04.417396    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-fcd9f3-29827590-dfxb7" event={"ID":"a92a7aa0-ad51-4c3c-a0b0-a25972496d3a","Type":"ContainerDied","Data":"a5e3b4f61495adcfd9b64f4edc15243d47a031b62cbca45df510d279caafc604"}
Sep 17 14:30:04.417561 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:04.417446    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a5e3b4f61495adcfd9b64f4edc15243d47a031b62cbca45df510d279caafc604"
Sep 17 14:30:04.417561 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:04.417402    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-fcd9f3-29827590-dfxb7"
Sep 17 14:30:04.418825 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:04.418800    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29827590-ncd6j"
Sep 17 14:30:04.418934 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:04.418844    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29827590-ncd6j" event={"ID":"82b52f92-7f30-4db4-b1fc-b013003c0e13","Type":"ContainerDied","Data":"26c80cd9197d836bd56cc7914a295ca0bc5addd0fc1c463011b645f0e455e781"}
Sep 17 14:30:04.418934 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:04.418889    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="26c80cd9197d836bd56cc7914a295ca0bc5addd0fc1c463011b645f0e455e781"
Sep 17 14:30:04.420292 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:04.420267    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-1269b2-29827590-rph8m"
Sep 17 14:30:04.420388 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:04.420264    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-1269b2-29827590-rph8m" event={"ID":"c9a93782-7587-44b5-a57b-1e6522631a2a","Type":"ContainerDied","Data":"8499d84e429a7c1feac66edd938315f529b74f3910a5aec93a68dee55832d29c"}
Sep 17 14:30:04.420388 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:04.420382    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8499d84e429a7c1feac66edd938315f529b74f3910a5aec93a68dee55832d29c"
Sep 17 14:30:04.422254 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:04.422228    2582 generic.go:356] "Generic (PLEG): container finished" podID="1e111231-9395-400a-834f-e1e3a0215df3" containerID="29e47ec63ccc365eda39e8bbc6729b060249b6099a9c821190132119bfa5a67d" exitCode=0
Sep 17 14:30:04.422355 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:04.422258    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a954c8-29827590-r7j5l" event={"ID":"1e111231-9395-400a-834f-e1e3a0215df3","Type":"ContainerDied","Data":"29e47ec63ccc365eda39e8bbc6729b060249b6099a9c821190132119bfa5a67d"}
Sep 17 14:30:04.422355 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:04.422332    2582 scope.go:117] "RemoveContainer" containerID="d2c73eeac888155ca4c02534a08a4a4025b2dc937b39291e830c75cdf504e02c"
Sep 17 14:30:04.425324 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:04.423896    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-cc3801-29827590-96rhs" event={"ID":"f6d9f887-eea1-488c-abc4-01981a9e8724","Type":"ContainerDied","Data":"1137427fceee72b29f509e517a7c6073cb7092521971e73d70f2622f03f5d7f4"}
Sep 17 14:30:04.425324 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:04.423991    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1137427fceee72b29f509e517a7c6073cb7092521971e73d70f2622f03f5d7f4"
Sep 17 14:30:04.425324 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:04.423920    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-cc3801-29827590-96rhs"
Sep 17 14:30:05.173432 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:05.173394    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-1269b2/embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt"]
Sep 17 14:30:05.173981 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:05.173962    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9a93782-7587-44b5-a57b-1e6522631a2a" containerName="cleanup"
Sep 17 14:30:05.174031 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:05.173983    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9a93782-7587-44b5-a57b-1e6522631a2a" containerName="cleanup"
Sep 17 14:30:05.174031 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:05.173994    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6d9f887-eea1-488c-abc4-01981a9e8724" containerName="cleanup"
Sep 17 14:30:05.174031 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:05.173999    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6d9f887-eea1-488c-abc4-01981a9e8724" containerName="cleanup"
Sep 17 14:30:05.174031 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:05.174010    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82b52f92-7f30-4db4-b1fc-b013003c0e13" containerName="cleanup"
Sep 17 14:30:05.174031 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:05.174018    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="82b52f92-7f30-4db4-b1fc-b013003c0e13" containerName="cleanup"
Sep 17 14:30:05.174031 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:05.174030    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a376b4d-87db-4a35-b4a7-d0e34ab90b47" containerName="cleanup"
Sep 17 14:30:05.174235 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:05.174035    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a376b4d-87db-4a35-b4a7-d0e34ab90b47" containerName="cleanup"
Sep 17 14:30:05.174235 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:05.174044    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a92a7aa0-ad51-4c3c-a0b0-a25972496d3a" containerName="cleanup"
Sep 17 14:30:05.174235 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:05.174049    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a92a7aa0-ad51-4c3c-a0b0-a25972496d3a" containerName="cleanup"
Sep 17 14:30:05.174235 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:05.174121    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a376b4d-87db-4a35-b4a7-d0e34ab90b47" containerName="cleanup"
Sep 17 14:30:05.174235 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:05.174129    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9a93782-7587-44b5-a57b-1e6522631a2a" containerName="cleanup"
Sep 17 14:30:05.174235 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:05.174135    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6d9f887-eea1-488c-abc4-01981a9e8724" containerName="cleanup"
Sep 17 14:30:05.174235 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:05.174144    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="82b52f92-7f30-4db4-b1fc-b013003c0e13" containerName="cleanup"
Sep 17 14:30:05.174235 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:05.174150    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a92a7aa0-ad51-4c3c-a0b0-a25972496d3a" containerName="cleanup"
Sep 17 14:30:05.178635 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:05.178588    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-1269b2/embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt"
Sep 17 14:30:05.181211 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:05.181190    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-1269b2\"/\"openshift-service-ca.crt\""
Sep 17 14:30:05.182025 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:05.181996    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-1269b2\"/\"default-dockercfg-dsxwt\""
Sep 17 14:30:05.182130 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:05.182086    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-1269b2\"/\"embedding-w0-1269b2-kserve-self-signed-certs\""
Sep 17 14:30:05.182363 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:05.182332    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-1269b2\"/\"kube-root-ca.crt\""
Sep 17 14:30:05.185502 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:05.185481    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-1269b2/embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt"]
Sep 17 14:30:05.327114 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:05.327071    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31b949bb-4068-49d8-9df6-be2801c9eef9-dshm\") pod \"embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt\" (UID: \"31b949bb-4068-49d8-9df6-be2801c9eef9\") " pod="e2e-models-e2e-worker-w0-1269b2/embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt"
Sep 17 14:30:05.327330 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:05.327254    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31b949bb-4068-49d8-9df6-be2801c9eef9-model-cache\") pod \"embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt\" (UID: \"31b949bb-4068-49d8-9df6-be2801c9eef9\") " pod="e2e-models-e2e-worker-w0-1269b2/embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt"
Sep 17 14:30:05.327330 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:05.327309    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31b949bb-4068-49d8-9df6-be2801c9eef9-tls-certs\") pod \"embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt\" (UID: \"31b949bb-4068-49d8-9df6-be2801c9eef9\") " pod="e2e-models-e2e-worker-w0-1269b2/embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt"
Sep 17 14:30:05.327482 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:05.327345    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31b949bb-4068-49d8-9df6-be2801c9eef9-home\") pod \"embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt\" (UID: \"31b949bb-4068-49d8-9df6-be2801c9eef9\") " pod="e2e-models-e2e-worker-w0-1269b2/embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt"
Sep 17 14:30:05.327482 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:05.327404    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31b949bb-4068-49d8-9df6-be2801c9eef9-tmp-dir\") pod \"embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt\" (UID: \"31b949bb-4068-49d8-9df6-be2801c9eef9\") " pod="e2e-models-e2e-worker-w0-1269b2/embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt"
Sep 17 14:30:05.327594 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:05.327478    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwkpj\" (UniqueName: \"kubernetes.io/projected/31b949bb-4068-49d8-9df6-be2801c9eef9-kube-api-access-wwkpj\") pod \"embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt\" (UID: \"31b949bb-4068-49d8-9df6-be2801c9eef9\") " pod="e2e-models-e2e-worker-w0-1269b2/embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt"
Sep 17 14:30:05.428293 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:05.428199    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31b949bb-4068-49d8-9df6-be2801c9eef9-model-cache\") pod \"embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt\" (UID: \"31b949bb-4068-49d8-9df6-be2801c9eef9\") " pod="e2e-models-e2e-worker-w0-1269b2/embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt"
Sep 17 14:30:05.428293 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:05.428251    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31b949bb-4068-49d8-9df6-be2801c9eef9-tls-certs\") pod \"embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt\" (UID: \"31b949bb-4068-49d8-9df6-be2801c9eef9\") " pod="e2e-models-e2e-worker-w0-1269b2/embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt"
Sep 17 14:30:05.428293 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:05.428279    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31b949bb-4068-49d8-9df6-be2801c9eef9-home\") pod \"embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt\" (UID: \"31b949bb-4068-49d8-9df6-be2801c9eef9\") " pod="e2e-models-e2e-worker-w0-1269b2/embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt"
Sep 17 14:30:05.428904 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:05.428326    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31b949bb-4068-49d8-9df6-be2801c9eef9-tmp-dir\") pod \"embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt\" (UID: \"31b949bb-4068-49d8-9df6-be2801c9eef9\") " pod="e2e-models-e2e-worker-w0-1269b2/embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt"
Sep 17 14:30:05.428904 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:05.428383    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwkpj\" (UniqueName: \"kubernetes.io/projected/31b949bb-4068-49d8-9df6-be2801c9eef9-kube-api-access-wwkpj\") pod \"embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt\" (UID: \"31b949bb-4068-49d8-9df6-be2801c9eef9\") " pod="e2e-models-e2e-worker-w0-1269b2/embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt"
Sep 17 14:30:05.428904 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:05.428447    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31b949bb-4068-49d8-9df6-be2801c9eef9-dshm\") pod \"embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt\" (UID: \"31b949bb-4068-49d8-9df6-be2801c9eef9\") " pod="e2e-models-e2e-worker-w0-1269b2/embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt"
Sep 17 14:30:05.428904 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:05.428704    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31b949bb-4068-49d8-9df6-be2801c9eef9-model-cache\") pod \"embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt\" (UID: \"31b949bb-4068-49d8-9df6-be2801c9eef9\") " pod="e2e-models-e2e-worker-w0-1269b2/embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt"
Sep 17 14:30:05.429113 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:05.428960    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31b949bb-4068-49d8-9df6-be2801c9eef9-home\") pod \"embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt\" (UID: \"31b949bb-4068-49d8-9df6-be2801c9eef9\") " pod="e2e-models-e2e-worker-w0-1269b2/embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt"
Sep 17 14:30:05.429552 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:05.429531    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31b949bb-4068-49d8-9df6-be2801c9eef9-tmp-dir\") pod \"embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt\" (UID: \"31b949bb-4068-49d8-9df6-be2801c9eef9\") " pod="e2e-models-e2e-worker-w0-1269b2/embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt"
Sep 17 14:30:05.431398 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:05.431375    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31b949bb-4068-49d8-9df6-be2801c9eef9-dshm\") pod \"embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt\" (UID: \"31b949bb-4068-49d8-9df6-be2801c9eef9\") " pod="e2e-models-e2e-worker-w0-1269b2/embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt"
Sep 17 14:30:05.432168 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:05.432126    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31b949bb-4068-49d8-9df6-be2801c9eef9-tls-certs\") pod \"embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt\" (UID: \"31b949bb-4068-49d8-9df6-be2801c9eef9\") " pod="e2e-models-e2e-worker-w0-1269b2/embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt"
Sep 17 14:30:05.437481 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:05.437462    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwkpj\" (UniqueName: \"kubernetes.io/projected/31b949bb-4068-49d8-9df6-be2801c9eef9-kube-api-access-wwkpj\") pod \"embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt\" (UID: \"31b949bb-4068-49d8-9df6-be2801c9eef9\") " pod="e2e-models-e2e-worker-w0-1269b2/embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt"
Sep 17 14:30:05.489987 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:05.489746    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-1269b2/embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt"
Sep 17 14:30:05.604377 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:05.604340    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a954c8-29827590-r7j5l"
Sep 17 14:30:05.735476 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:05.735438    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:30:05.782601 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:05.782575    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-1269b2/embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt"]
Sep 17 14:30:05.784644 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:30:05.784599    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod31b949bb_4068_49d8_9df6_be2801c9eef9.slice/crio-9295d3e853bdeece55fa525bc975bdbdd24ce81623559c59c762063bda96cd4d WatchSource:0}: Error finding container 9295d3e853bdeece55fa525bc975bdbdd24ce81623559c59c762063bda96cd4d: Status 404 returned error can't find the container with id 9295d3e853bdeece55fa525bc975bdbdd24ce81623559c59c762063bda96cd4d
Sep 17 14:30:06.451619 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:06.451565    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a954c8-29827590-r7j5l"
Sep 17 14:30:06.452074 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:06.451573    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a954c8-29827590-r7j5l" event={"ID":"1e111231-9395-400a-834f-e1e3a0215df3","Type":"ContainerDied","Data":"d4c4658983c179b98dbe14cd1a2678770f38397c548d94cd56bacd6330617192"}
Sep 17 14:30:06.452074 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:06.451727    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d4c4658983c179b98dbe14cd1a2678770f38397c548d94cd56bacd6330617192"
Sep 17 14:30:06.453476 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:06.453445    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-1269b2/embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt" event={"ID":"31b949bb-4068-49d8-9df6-be2801c9eef9","Type":"ContainerStarted","Data":"d70248c1e082876843e90b7d2b35e8ce1b983807e0e51abf38275d3a065cd3b0"}
Sep 17 14:30:06.453624 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:06.453488    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-1269b2/embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt" event={"ID":"31b949bb-4068-49d8-9df6-be2801c9eef9","Type":"ContainerStarted","Data":"9295d3e853bdeece55fa525bc975bdbdd24ce81623559c59c762063bda96cd4d"}
Sep 17 14:30:06.472063 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:06.472016    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-1269b2/embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt" podStartSLOduration=1.472001155 podStartE2EDuration="1.472001155s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:30:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:30:06.471150569 +0000 UTC m=+1507.472694045" watchObservedRunningTime="2026-09-17 14:30:06.472001155 +0000 UTC m=+1507.473544629"
Sep 17 14:30:13.520263 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:13.520192    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:30:15.495709 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:15.495672    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-1269b2/embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt"
Sep 17 14:30:15.496178 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:15.495722    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-1269b2/embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt"
Sep 17 14:30:15.504358 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:15.504333    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-1269b2/embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt"
Sep 17 14:30:15.525539 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:15.525506    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-1269b2/embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt"
Sep 17 14:30:16.018394 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:16.018356    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:30:17.283577 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:17.283537    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:30:17.955386 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:17.955349    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:30:18.842495 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:18.842454    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:30:19.547751 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:19.547713    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:30:20.251367 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:20.251330    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:30:20.326797 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:20.326759    2582 patch_prober.go:28] interesting pod/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 17 14:30:20.327003 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:20.326844    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv" podUID="7be60b88-b959-4272-a784-e6e96ab10098" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 17 14:30:20.349708 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:20.349673    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:30:20.811207 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:20.811169    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:30:21.044807 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.044767    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:30:21.092517 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.092437    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb"]
Sep 17 14:30:21.093241 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.093220    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e111231-9395-400a-834f-e1e3a0215df3" containerName="cleanup"
Sep 17 14:30:21.093293 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.093246    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e111231-9395-400a-834f-e1e3a0215df3" containerName="cleanup"
Sep 17 14:30:21.093332 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.093299    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e111231-9395-400a-834f-e1e3a0215df3" containerName="cleanup"
Sep 17 14:30:21.093332 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.093309    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e111231-9395-400a-834f-e1e3a0215df3" containerName="cleanup"
Sep 17 14:30:21.093453 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.093440    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e111231-9395-400a-834f-e1e3a0215df3" containerName="cleanup"
Sep 17 14:30:21.093489 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.093462    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e111231-9395-400a-834f-e1e3a0215df3" containerName="cleanup"
Sep 17 14:30:21.097527 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.097501    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb"
Sep 17 14:30:21.100175 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.100149    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-cf94fd-openshift-default-dockercfg-tq64c\""
Sep 17 14:30:21.106427 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.106399    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb"]
Sep 17 14:30:21.166376 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.166344    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:30:21.223368 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.220267    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sfhv\" (UniqueName: \"kubernetes.io/projected/225ff0fb-8843-44e7-a8c1-67a86eebcf25-kube-api-access-5sfhv\") pod \"e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb\" (UID: \"225ff0fb-8843-44e7-a8c1-67a86eebcf25\") " pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb"
Sep 17 14:30:21.223368 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.220337    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/225ff0fb-8843-44e7-a8c1-67a86eebcf25-istio-token\") pod \"e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb\" (UID: \"225ff0fb-8843-44e7-a8c1-67a86eebcf25\") " pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb"
Sep 17 14:30:21.223368 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.220401    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/225ff0fb-8843-44e7-a8c1-67a86eebcf25-istio-envoy\") pod \"e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb\" (UID: \"225ff0fb-8843-44e7-a8c1-67a86eebcf25\") " pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb"
Sep 17 14:30:21.223368 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.220448    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/225ff0fb-8843-44e7-a8c1-67a86eebcf25-istiod-ca-cert\") pod \"e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb\" (UID: \"225ff0fb-8843-44e7-a8c1-67a86eebcf25\") " pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb"
Sep 17 14:30:21.223368 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.220511    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/225ff0fb-8843-44e7-a8c1-67a86eebcf25-istio-podinfo\") pod \"e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb\" (UID: \"225ff0fb-8843-44e7-a8c1-67a86eebcf25\") " pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb"
Sep 17 14:30:21.223368 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.220560    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/225ff0fb-8843-44e7-a8c1-67a86eebcf25-istio-data\") pod \"e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb\" (UID: \"225ff0fb-8843-44e7-a8c1-67a86eebcf25\") " pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb"
Sep 17 14:30:21.223368 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.220677    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/225ff0fb-8843-44e7-a8c1-67a86eebcf25-credential-socket\") pod \"e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb\" (UID: \"225ff0fb-8843-44e7-a8c1-67a86eebcf25\") " pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb"
Sep 17 14:30:21.223368 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.220739    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/225ff0fb-8843-44e7-a8c1-67a86eebcf25-workload-socket\") pod \"e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb\" (UID: \"225ff0fb-8843-44e7-a8c1-67a86eebcf25\") " pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb"
Sep 17 14:30:21.223368 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.220770    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/225ff0fb-8843-44e7-a8c1-67a86eebcf25-workload-certs\") pod \"e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb\" (UID: \"225ff0fb-8843-44e7-a8c1-67a86eebcf25\") " pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb"
Sep 17 14:30:21.321899 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.321862    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/225ff0fb-8843-44e7-a8c1-67a86eebcf25-istio-data\") pod \"e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb\" (UID: \"225ff0fb-8843-44e7-a8c1-67a86eebcf25\") " pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb"
Sep 17 14:30:21.321899 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.321904    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/225ff0fb-8843-44e7-a8c1-67a86eebcf25-credential-socket\") pod \"e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb\" (UID: \"225ff0fb-8843-44e7-a8c1-67a86eebcf25\") " pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb"
Sep 17 14:30:21.322408 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.321922    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/225ff0fb-8843-44e7-a8c1-67a86eebcf25-workload-socket\") pod \"e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb\" (UID: \"225ff0fb-8843-44e7-a8c1-67a86eebcf25\") " pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb"
Sep 17 14:30:21.322408 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.322051    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/225ff0fb-8843-44e7-a8c1-67a86eebcf25-workload-certs\") pod \"e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb\" (UID: \"225ff0fb-8843-44e7-a8c1-67a86eebcf25\") " pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb"
Sep 17 14:30:21.322408 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.322157    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sfhv\" (UniqueName: \"kubernetes.io/projected/225ff0fb-8843-44e7-a8c1-67a86eebcf25-kube-api-access-5sfhv\") pod \"e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb\" (UID: \"225ff0fb-8843-44e7-a8c1-67a86eebcf25\") " pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb"
Sep 17 14:30:21.322408 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.322214    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/225ff0fb-8843-44e7-a8c1-67a86eebcf25-istio-token\") pod \"e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb\" (UID: \"225ff0fb-8843-44e7-a8c1-67a86eebcf25\") " pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb"
Sep 17 14:30:21.322408 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.322295    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/225ff0fb-8843-44e7-a8c1-67a86eebcf25-istio-envoy\") pod \"e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb\" (UID: \"225ff0fb-8843-44e7-a8c1-67a86eebcf25\") " pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb"
Sep 17 14:30:21.322408 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.322355    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/225ff0fb-8843-44e7-a8c1-67a86eebcf25-istiod-ca-cert\") pod \"e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb\" (UID: \"225ff0fb-8843-44e7-a8c1-67a86eebcf25\") " pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb"
Sep 17 14:30:21.322408 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.322382    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/225ff0fb-8843-44e7-a8c1-67a86eebcf25-workload-socket\") pod \"e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb\" (UID: \"225ff0fb-8843-44e7-a8c1-67a86eebcf25\") " pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb"
Sep 17 14:30:21.322798 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.322433    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/225ff0fb-8843-44e7-a8c1-67a86eebcf25-istio-data\") pod \"e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb\" (UID: \"225ff0fb-8843-44e7-a8c1-67a86eebcf25\") " pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb"
Sep 17 14:30:21.322798 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.322445    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/225ff0fb-8843-44e7-a8c1-67a86eebcf25-istio-podinfo\") pod \"e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb\" (UID: \"225ff0fb-8843-44e7-a8c1-67a86eebcf25\") " pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb"
Sep 17 14:30:21.322798 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.322460    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/225ff0fb-8843-44e7-a8c1-67a86eebcf25-workload-certs\") pod \"e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb\" (UID: \"225ff0fb-8843-44e7-a8c1-67a86eebcf25\") " pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb"
Sep 17 14:30:21.322798 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.322648    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/225ff0fb-8843-44e7-a8c1-67a86eebcf25-credential-socket\") pod \"e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb\" (UID: \"225ff0fb-8843-44e7-a8c1-67a86eebcf25\") " pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb"
Sep 17 14:30:21.323115 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.323093    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/225ff0fb-8843-44e7-a8c1-67a86eebcf25-istiod-ca-cert\") pod \"e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb\" (UID: \"225ff0fb-8843-44e7-a8c1-67a86eebcf25\") " pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb"
Sep 17 14:30:21.324760 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.324732    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/225ff0fb-8843-44e7-a8c1-67a86eebcf25-istio-envoy\") pod \"e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb\" (UID: \"225ff0fb-8843-44e7-a8c1-67a86eebcf25\") " pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb"
Sep 17 14:30:21.324873 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.324844    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/225ff0fb-8843-44e7-a8c1-67a86eebcf25-istio-podinfo\") pod \"e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb\" (UID: \"225ff0fb-8843-44e7-a8c1-67a86eebcf25\") " pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb"
Sep 17 14:30:21.330470 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.330446    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sfhv\" (UniqueName: \"kubernetes.io/projected/225ff0fb-8843-44e7-a8c1-67a86eebcf25-kube-api-access-5sfhv\") pod \"e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb\" (UID: \"225ff0fb-8843-44e7-a8c1-67a86eebcf25\") " pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb"
Sep 17 14:30:21.330965 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.330863    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/225ff0fb-8843-44e7-a8c1-67a86eebcf25-istio-token\") pod \"e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb\" (UID: \"225ff0fb-8843-44e7-a8c1-67a86eebcf25\") " pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb"
Sep 17 14:30:21.412476 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.412437    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb"
Sep 17 14:30:21.416111 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.416072    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:30:21.558595 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.558564    2582 generic.go:356] "Generic (PLEG): container finished" podID="7be60b88-b959-4272-a784-e6e96ab10098" containerID="452eadf230bb93dd03220d6cc458c409dbc6952adc525b7d1569de2abad8805c" exitCode=0
Sep 17 14:30:21.558773 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.558649    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv" event={"ID":"7be60b88-b959-4272-a784-e6e96ab10098","Type":"ContainerDied","Data":"452eadf230bb93dd03220d6cc458c409dbc6952adc525b7d1569de2abad8805c"}
Sep 17 14:30:21.559130 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.559112    2582 scope.go:117] "RemoveContainer" containerID="452eadf230bb93dd03220d6cc458c409dbc6952adc525b7d1569de2abad8805c"
Sep 17 14:30:21.560439 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.560413    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:30:21.560515 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.560481    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:30:21.560559 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.560527    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:30:21.692989 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.692958    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb"]
Sep 17 14:30:21.696343 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:30:21.696311    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod225ff0fb_8843_44e7_a8c1_67a86eebcf25.slice/crio-734a2201d1f3494b69ec9871f5821e6d1cb73f03ae3055c544916aa346a0d96f WatchSource:0}: Error finding container 734a2201d1f3494b69ec9871f5821e6d1cb73f03ae3055c544916aa346a0d96f: Status 404 returned error can't find the container with id 734a2201d1f3494b69ec9871f5821e6d1cb73f03ae3055c544916aa346a0d96f
Sep 17 14:30:21.698962 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.698926    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:30:21.699070 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.699011    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:30:21.699070 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.699057    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:30:21.734036 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:21.733948    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:30:22.506322 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:22.506277    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:30:22.572214 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:22.572115    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv" event={"ID":"7be60b88-b959-4272-a784-e6e96ab10098","Type":"ContainerStarted","Data":"4f279aa4f4f3a499f29c7a35722217c2ff6f5d790575019074a984440d281c03"}
Sep 17 14:30:22.574966 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:22.574932    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb" event={"ID":"225ff0fb-8843-44e7-a8c1-67a86eebcf25","Type":"ContainerStarted","Data":"f6a0fcc25a0b82a2bd124bbed88079b8076d60f8ba56c058f0ba955dc43456ae"}
Sep 17 14:30:22.575189 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:22.575170    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb" event={"ID":"225ff0fb-8843-44e7-a8c1-67a86eebcf25","Type":"ContainerStarted","Data":"734a2201d1f3494b69ec9871f5821e6d1cb73f03ae3055c544916aa346a0d96f"}
Sep 17 14:30:22.619184 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:22.619139    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb" podStartSLOduration=1.619123211 podStartE2EDuration="1.619123211s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:30:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:30:22.617007347 +0000 UTC m=+1523.618550822" watchObservedRunningTime="2026-09-17 14:30:22.619123211 +0000 UTC m=+1523.620666684"
Sep 17 14:30:22.980527 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:22.980496    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv"
Sep 17 14:30:22.980792 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:22.980769    2582 patch_prober.go:28] interesting pod/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Sep 17 14:30:22.980912 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:22.980828    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv" podUID="7be60b88-b959-4272-a784-e6e96ab10098" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Sep 17 14:30:23.412967 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:23.412864    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb"
Sep 17 14:30:23.419138 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:23.419110    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb"
Sep 17 14:30:23.582656 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:23.582599    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb"
Sep 17 14:30:23.583809 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:23.583786    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb"
Sep 17 14:30:23.944260 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:23.944213    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:30:23.980435 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:23.980400    2582 patch_prober.go:28] interesting pod/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Sep 17 14:30:23.980641 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:23.980464    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv" podUID="7be60b88-b959-4272-a784-e6e96ab10098" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Sep 17 14:30:24.546756 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:24.546713    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:30:24.647279 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:24.647222    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:30:24.746805 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:24.746752    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:30:24.846282 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:24.846183    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:30:24.979966 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:24.979930    2582 patch_prober.go:28] interesting pod/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Sep 17 14:30:24.980148 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:24.979997    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv" podUID="7be60b88-b959-4272-a784-e6e96ab10098" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Sep 17 14:30:25.643295 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:25.643240    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:30:25.980027 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:25.979988    2582 patch_prober.go:28] interesting pod/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Sep 17 14:30:25.980503 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:25.980054    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv" podUID="7be60b88-b959-4272-a784-e6e96ab10098" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Sep 17 14:30:26.604281 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:26.604250    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:30:26.979930 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:26.979895    2582 patch_prober.go:28] interesting pod/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Sep 17 14:30:26.980108 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:26.979951    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv" podUID="7be60b88-b959-4272-a784-e6e96ab10098" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Sep 17 14:30:27.223765 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:27.223726    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:30:27.980702 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:27.980661    2582 patch_prober.go:28] interesting pod/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Sep 17 14:30:27.981099 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:27.980734    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv" podUID="7be60b88-b959-4272-a784-e6e96ab10098" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Sep 17 14:30:28.045177 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:28.045129    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:30:28.893209 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:28.893173    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:30:28.983445 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:28.983412    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv"
Sep 17 14:30:28.983939 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:28.983841    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv"
Sep 17 14:30:28.984418 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:28.984398    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv"
Sep 17 14:30:29.524251 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:29.524201    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:30:31.074128 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:31.074082    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:30:52.149748 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:52.149696    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:30:53.133665 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:53.133625    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:30:53.870757 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:53.870702    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:30:54.731991 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:54.731956    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:30:54.921456 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:54.921415    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:30:55.667343 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:55.667289    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:30:56.341686 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:56.341646    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:30:57.026478 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:57.026432    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:30:57.778501 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:57.778466    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a954c8-7q8m6"]
Sep 17 14:30:57.781228 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:57.781210    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a954c8-7q8m6"
Sep 17 14:30:57.783570 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:57.783548    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ddwrn\""
Sep 17 14:30:57.790529 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:57.790505    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a954c8-7q8m6"]
Sep 17 14:30:57.819522 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:57.819489    2582 generic.go:356] "Generic (PLEG): container finished" podID="225ff0fb-8843-44e7-a8c1-67a86eebcf25" containerID="f6a0fcc25a0b82a2bd124bbed88079b8076d60f8ba56c058f0ba955dc43456ae" exitCode=0
Sep 17 14:30:57.819680 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:57.819566    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb" event={"ID":"225ff0fb-8843-44e7-a8c1-67a86eebcf25","Type":"ContainerDied","Data":"f6a0fcc25a0b82a2bd124bbed88079b8076d60f8ba56c058f0ba955dc43456ae"}
Sep 17 14:30:57.820174 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:57.820153    2582 scope.go:117] "RemoveContainer" containerID="f6a0fcc25a0b82a2bd124bbed88079b8076d60f8ba56c058f0ba955dc43456ae"
Sep 17 14:30:57.821497 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:57.821469    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:30:57.821587 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:57.821529    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:30:57.821587 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:57.821564    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 14:30:57.847797 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:57.847765    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb4e8661-72e4-4115-8675-e26f88aa607e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-a954c8-7q8m6\" (UID: \"bb4e8661-72e4-4115-8675-e26f88aa607e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a954c8-7q8m6"
Sep 17 14:30:57.949213 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:57.949181    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb4e8661-72e4-4115-8675-e26f88aa607e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-a954c8-7q8m6\" (UID: \"bb4e8661-72e4-4115-8675-e26f88aa607e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a954c8-7q8m6"
Sep 17 14:30:57.949822 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:57.949801    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb4e8661-72e4-4115-8675-e26f88aa607e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-a954c8-7q8m6\" (UID: \"bb4e8661-72e4-4115-8675-e26f88aa607e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a954c8-7q8m6"
Sep 17 14:30:57.960516 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:57.960490    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:30:58.094121 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:58.094038    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a954c8-7q8m6"
Sep 17 14:30:58.372231 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:58.372068    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a954c8-7q8m6"]
Sep 17 14:30:58.373841 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:30:58.373808    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb4e8661_72e4_4115_8675_e26f88aa607e.slice/crio-41a34e98ea228f72a983073331dcf873e61aebd90994cee65c5136968fd5758c WatchSource:0}: Error finding container 41a34e98ea228f72a983073331dcf873e61aebd90994cee65c5136968fd5758c: Status 404 returned error can't find the container with id 41a34e98ea228f72a983073331dcf873e61aebd90994cee65c5136968fd5758c
Sep 17 14:30:58.652633 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:58.650283    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:30:58.829693 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:58.828587    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb" event={"ID":"225ff0fb-8843-44e7-a8c1-67a86eebcf25","Type":"ContainerStarted","Data":"8aeaca45a8f40044d7ba9a9285231243daf5b4b1aa1a2e50531c460cd0ea5cd4"}
Sep 17 14:30:58.833067 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:58.832245    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a954c8-7q8m6" event={"ID":"bb4e8661-72e4-4115-8675-e26f88aa607e","Type":"ContainerStarted","Data":"f47d9fedc866c61d8316df21ebe9f7adb3bfd28469f925ec7af0ec5ccba812fd"}
Sep 17 14:30:58.833067 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:58.832298    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a954c8-7q8m6" event={"ID":"bb4e8661-72e4-4115-8675-e26f88aa607e","Type":"ContainerStarted","Data":"41a34e98ea228f72a983073331dcf873e61aebd90994cee65c5136968fd5758c"}
Sep 17 14:30:58.866391 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:58.866335    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a954c8-7q8m6" podStartSLOduration=1.86631813 podStartE2EDuration="1.86631813s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:30:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:30:58.864946507 +0000 UTC m=+1559.866489984" watchObservedRunningTime="2026-09-17 14:30:58.86631813 +0000 UTC m=+1559.867861604"
Sep 17 14:30:59.359381 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:59.359350    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:30:59.413017 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:59.412992    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb"
Sep 17 14:30:59.413196 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:59.413177    2582 patch_prober.go:28] interesting pod/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Sep 17 14:30:59.413252 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:59.413226    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb" podUID="225ff0fb-8843-44e7-a8c1-67a86eebcf25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Sep 17 14:30:59.845051 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:59.845008    2582 generic.go:356] "Generic (PLEG): container finished" podID="bb4e8661-72e4-4115-8675-e26f88aa607e" containerID="f47d9fedc866c61d8316df21ebe9f7adb3bfd28469f925ec7af0ec5ccba812fd" exitCode=0
Sep 17 14:30:59.845544 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:30:59.845083    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a954c8-7q8m6" event={"ID":"bb4e8661-72e4-4115-8675-e26f88aa607e","Type":"ContainerDied","Data":"f47d9fedc866c61d8316df21ebe9f7adb3bfd28469f925ec7af0ec5ccba812fd"}
Sep 17 14:31:00.192675 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:00.192577    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:00.412910 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:00.412873    2582 patch_prober.go:28] interesting pod/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Sep 17 14:31:00.413108 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:00.412931    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb" podUID="225ff0fb-8843-44e7-a8c1-67a86eebcf25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Sep 17 14:31:00.942643 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:00.940145    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:01.072436 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:01.072405    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a954c8-7q8m6"
Sep 17 14:31:01.079360 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:01.079339    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb4e8661-72e4-4115-8675-e26f88aa607e-maas-api-service-ca\") pod \"bb4e8661-72e4-4115-8675-e26f88aa607e\" (UID: \"bb4e8661-72e4-4115-8675-e26f88aa607e\") "
Sep 17 14:31:01.079687 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:01.079666    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb4e8661-72e4-4115-8675-e26f88aa607e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bb4e8661-72e4-4115-8675-e26f88aa607e" (UID: "bb4e8661-72e4-4115-8675-e26f88aa607e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:31:01.181062 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:01.180963    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb4e8661-72e4-4115-8675-e26f88aa607e-maas-api-service-ca\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:31:01.413137 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:01.413098    2582 patch_prober.go:28] interesting pod/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Sep 17 14:31:01.413317 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:01.413159    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb" podUID="225ff0fb-8843-44e7-a8c1-67a86eebcf25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Sep 17 14:31:01.791438 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:01.791331    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:01.862832 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:01.862803    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a954c8-7q8m6"
Sep 17 14:31:01.863003 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:01.862832    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a954c8-7q8m6" event={"ID":"bb4e8661-72e4-4115-8675-e26f88aa607e","Type":"ContainerDied","Data":"41a34e98ea228f72a983073331dcf873e61aebd90994cee65c5136968fd5758c"}
Sep 17 14:31:01.863003 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:01.862879    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="41a34e98ea228f72a983073331dcf873e61aebd90994cee65c5136968fd5758c"
Sep 17 14:31:02.398089 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:02.398057    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:02.412784 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:02.412755    2582 patch_prober.go:28] interesting pod/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Sep 17 14:31:02.412930 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:02.412814    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb" podUID="225ff0fb-8843-44e7-a8c1-67a86eebcf25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Sep 17 14:31:03.160193 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:03.160123    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:03.412994 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:03.412903    2582 patch_prober.go:28] interesting pod/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Sep 17 14:31:03.412994 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:03.412960    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb" podUID="225ff0fb-8843-44e7-a8c1-67a86eebcf25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Sep 17 14:31:03.932397 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:03.932350    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:04.417215 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:04.417184    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb"
Sep 17 14:31:04.417643 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:04.417602    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb"
Sep 17 14:31:04.418351 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:04.418331    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb"
Sep 17 14:31:05.177853 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:05.177817    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:08.035416 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:08.035377    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:08.872864 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:08.872830    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:13.636464 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:13.636426    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a954c8-29827590-r7j5l"]
Sep 17 14:31:13.639254 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:13.639226    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a954c8-29827590-r7j5l"]
Sep 17 14:31:13.650303 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:13.650278    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:13.751705 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:13.751672    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:13.844649 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:13.844594    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a954c8-7q8m6"]
Sep 17 14:31:13.848220 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:13.848184    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:13.848373 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:13.848240    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a954c8-7q8m6"]
Sep 17 14:31:13.928092 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:13.927969    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:14.237697 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:14.237659    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:14.346260 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:14.346195    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:14.986966 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:14.986935    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:15.495139 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:15.495102    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e111231-9395-400a-834f-e1e3a0215df3" path="/var/lib/kubelet/pods/1e111231-9395-400a-834f-e1e3a0215df3/volumes"
Sep 17 14:31:15.495524 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:15.495509    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb4e8661-72e4-4115-8675-e26f88aa607e" path="/var/lib/kubelet/pods/bb4e8661-72e4-4115-8675-e26f88aa607e/volumes"
Sep 17 14:31:15.727679 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:15.727595    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:16.416720 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:16.416655    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:17.051560 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:17.051514    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:17.872132 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:17.872099    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:18.600516 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:18.600481    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:19.297662 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:19.297602    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:20.443388 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:20.443353    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:20.547540 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:20.547511    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:20.657269 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:20.657236    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv"]
Sep 17 14:31:20.657461 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:20.657439    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv" podUID="7be60b88-b959-4272-a784-e6e96ab10098" containerName="istio-proxy" containerID="cri-o://4f279aa4f4f3a499f29c7a35722217c2ff6f5d790575019074a984440d281c03" gracePeriod=30
Sep 17 14:31:20.703867 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:20.703784    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:21.027843 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:21.027757    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cf94fd-mqcp9"]
Sep 17 14:31:21.028520 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:21.028501    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb4e8661-72e4-4115-8675-e26f88aa607e" containerName="revoke-keys"
Sep 17 14:31:21.028565 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:21.028526    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb4e8661-72e4-4115-8675-e26f88aa607e" containerName="revoke-keys"
Sep 17 14:31:21.028700 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:21.028686    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb4e8661-72e4-4115-8675-e26f88aa607e" containerName="revoke-keys"
Sep 17 14:31:21.031949 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:21.031931    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cf94fd-mqcp9"
Sep 17 14:31:21.034451 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:21.034428    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ddwrn\""
Sep 17 14:31:21.078434 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:21.078399    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cf94fd-mqcp9"]
Sep 17 14:31:21.103499 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:21.103462    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9963b26f-c2cb-4019-814f-869e4e53406c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-cf94fd-mqcp9\" (UID: \"9963b26f-c2cb-4019-814f-869e4e53406c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cf94fd-mqcp9"
Sep 17 14:31:21.204891 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:21.204857    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9963b26f-c2cb-4019-814f-869e4e53406c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-cf94fd-mqcp9\" (UID: \"9963b26f-c2cb-4019-814f-869e4e53406c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cf94fd-mqcp9"
Sep 17 14:31:21.205497 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:21.205477    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9963b26f-c2cb-4019-814f-869e4e53406c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-cf94fd-mqcp9\" (UID: \"9963b26f-c2cb-4019-814f-869e4e53406c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cf94fd-mqcp9"
Sep 17 14:31:21.342413 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:21.342316    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cf94fd-mqcp9"
Sep 17 14:31:21.351023 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:21.350989    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:21.614925 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:21.614891    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cf94fd-mqcp9"]
Sep 17 14:31:21.616208 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:31:21.616179    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9963b26f_c2cb_4019_814f_869e4e53406c.slice/crio-5ef2fae9c41cbe20f9d21921528392dda29bef879c9ebd731bad97b72286fe4c WatchSource:0}: Error finding container 5ef2fae9c41cbe20f9d21921528392dda29bef879c9ebd731bad97b72286fe4c: Status 404 returned error can't find the container with id 5ef2fae9c41cbe20f9d21921528392dda29bef879c9ebd731bad97b72286fe4c
Sep 17 14:31:22.009860 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:22.009821    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cf94fd-mqcp9" event={"ID":"9963b26f-c2cb-4019-814f-869e4e53406c","Type":"ContainerStarted","Data":"c27bfc8db786168ee575fc8a8f383ca1f07878f684db337d887d697db4abc159"}
Sep 17 14:31:22.010049 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:22.009879    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cf94fd-mqcp9" event={"ID":"9963b26f-c2cb-4019-814f-869e4e53406c","Type":"ContainerStarted","Data":"5ef2fae9c41cbe20f9d21921528392dda29bef879c9ebd731bad97b72286fe4c"}
Sep 17 14:31:22.028051 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:22.027993    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cf94fd-mqcp9" podStartSLOduration=1.027976501 podStartE2EDuration="1.027976501s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:31:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:31:22.026521898 +0000 UTC m=+1583.028065373" watchObservedRunningTime="2026-09-17 14:31:22.027976501 +0000 UTC m=+1583.029520003"
Sep 17 14:31:22.176291 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:22.176258    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:22.841304 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:22.841267    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:23.017309 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:23.017270    2582 generic.go:356] "Generic (PLEG): container finished" podID="9963b26f-c2cb-4019-814f-869e4e53406c" containerID="c27bfc8db786168ee575fc8a8f383ca1f07878f684db337d887d697db4abc159" exitCode=0
Sep 17 14:31:23.017476 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:23.017350    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cf94fd-mqcp9" event={"ID":"9963b26f-c2cb-4019-814f-869e4e53406c","Type":"ContainerDied","Data":"c27bfc8db786168ee575fc8a8f383ca1f07878f684db337d887d697db4abc159"}
Sep 17 14:31:23.529804 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:23.529753    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:24.171592 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:24.171568    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cf94fd-mqcp9"
Sep 17 14:31:24.318668 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:24.318563    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:24.348280 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:24.348227    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9963b26f-c2cb-4019-814f-869e4e53406c-maas-api-service-ca\") pod \"9963b26f-c2cb-4019-814f-869e4e53406c\" (UID: \"9963b26f-c2cb-4019-814f-869e4e53406c\") "
Sep 17 14:31:24.348758 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:24.348729    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9963b26f-c2cb-4019-814f-869e4e53406c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9963b26f-c2cb-4019-814f-869e4e53406c" (UID: "9963b26f-c2cb-4019-814f-869e4e53406c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:31:24.450127 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:24.450088    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9963b26f-c2cb-4019-814f-869e4e53406c-maas-api-service-ca\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:31:24.986931 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:24.986901    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:25.031778 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:25.031749    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cf94fd-mqcp9"
Sep 17 14:31:25.031956 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:25.031785    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cf94fd-mqcp9" event={"ID":"9963b26f-c2cb-4019-814f-869e4e53406c","Type":"ContainerDied","Data":"5ef2fae9c41cbe20f9d21921528392dda29bef879c9ebd731bad97b72286fe4c"}
Sep 17 14:31:25.031956 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:25.031841    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5ef2fae9c41cbe20f9d21921528392dda29bef879c9ebd731bad97b72286fe4c"
Sep 17 14:31:25.760912 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:25.760871    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:25.914157 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:25.914121    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv"
Sep 17 14:31:26.040909 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:26.040873    2582 generic.go:356] "Generic (PLEG): container finished" podID="7be60b88-b959-4272-a784-e6e96ab10098" containerID="4f279aa4f4f3a499f29c7a35722217c2ff6f5d790575019074a984440d281c03" exitCode=0
Sep 17 14:31:26.041129 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:26.040929    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv"
Sep 17 14:31:26.041129 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:26.040960    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv" event={"ID":"7be60b88-b959-4272-a784-e6e96ab10098","Type":"ContainerDied","Data":"4f279aa4f4f3a499f29c7a35722217c2ff6f5d790575019074a984440d281c03"}
Sep 17 14:31:26.041129 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:26.041029    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv" event={"ID":"7be60b88-b959-4272-a784-e6e96ab10098","Type":"ContainerDied","Data":"77aa303a62c48ac5afbdb08ae3750ffc552fd8795bc7e05c8885ad699229bd52"}
Sep 17 14:31:26.041129 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:26.041058    2582 scope.go:117] "RemoveContainer" containerID="4f279aa4f4f3a499f29c7a35722217c2ff6f5d790575019074a984440d281c03"
Sep 17 14:31:26.051121 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:26.051103    2582 scope.go:117] "RemoveContainer" containerID="452eadf230bb93dd03220d6cc458c409dbc6952adc525b7d1569de2abad8805c"
Sep 17 14:31:26.060290 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:26.060271    2582 scope.go:117] "RemoveContainer" containerID="4f279aa4f4f3a499f29c7a35722217c2ff6f5d790575019074a984440d281c03"
Sep 17 14:31:26.060560 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:31:26.060544    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f279aa4f4f3a499f29c7a35722217c2ff6f5d790575019074a984440d281c03\": container with ID starting with 4f279aa4f4f3a499f29c7a35722217c2ff6f5d790575019074a984440d281c03 not found: ID does not exist" containerID="4f279aa4f4f3a499f29c7a35722217c2ff6f5d790575019074a984440d281c03"
Sep 17 14:31:26.060700 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:26.060567    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f279aa4f4f3a499f29c7a35722217c2ff6f5d790575019074a984440d281c03"} err="failed to get container status \"4f279aa4f4f3a499f29c7a35722217c2ff6f5d790575019074a984440d281c03\": rpc error: code = NotFound desc = could not find container \"4f279aa4f4f3a499f29c7a35722217c2ff6f5d790575019074a984440d281c03\": container with ID starting with 4f279aa4f4f3a499f29c7a35722217c2ff6f5d790575019074a984440d281c03 not found: ID does not exist"
Sep 17 14:31:26.060700 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:26.060582    2582 scope.go:117] "RemoveContainer" containerID="452eadf230bb93dd03220d6cc458c409dbc6952adc525b7d1569de2abad8805c"
Sep 17 14:31:26.060837 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:31:26.060813    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"452eadf230bb93dd03220d6cc458c409dbc6952adc525b7d1569de2abad8805c\": container with ID starting with 452eadf230bb93dd03220d6cc458c409dbc6952adc525b7d1569de2abad8805c not found: ID does not exist" containerID="452eadf230bb93dd03220d6cc458c409dbc6952adc525b7d1569de2abad8805c"
Sep 17 14:31:26.060936 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:26.060838    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"452eadf230bb93dd03220d6cc458c409dbc6952adc525b7d1569de2abad8805c"} err="failed to get container status \"452eadf230bb93dd03220d6cc458c409dbc6952adc525b7d1569de2abad8805c\": rpc error: code = NotFound desc = could not find container \"452eadf230bb93dd03220d6cc458c409dbc6952adc525b7d1569de2abad8805c\": container with ID starting with 452eadf230bb93dd03220d6cc458c409dbc6952adc525b7d1569de2abad8805c not found: ID does not exist"
Sep 17 14:31:26.067207 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:26.067185    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7be60b88-b959-4272-a784-e6e96ab10098-istio-podinfo\") pod \"7be60b88-b959-4272-a784-e6e96ab10098\" (UID: \"7be60b88-b959-4272-a784-e6e96ab10098\") "
Sep 17 14:31:26.067296 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:26.067242    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7be60b88-b959-4272-a784-e6e96ab10098-workload-certs\") pod \"7be60b88-b959-4272-a784-e6e96ab10098\" (UID: \"7be60b88-b959-4272-a784-e6e96ab10098\") "
Sep 17 14:31:26.067296 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:26.067283    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7be60b88-b959-4272-a784-e6e96ab10098-istiod-ca-cert\") pod \"7be60b88-b959-4272-a784-e6e96ab10098\" (UID: \"7be60b88-b959-4272-a784-e6e96ab10098\") "
Sep 17 14:31:26.067375 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:26.067325    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7be60b88-b959-4272-a784-e6e96ab10098-workload-socket\") pod \"7be60b88-b959-4272-a784-e6e96ab10098\" (UID: \"7be60b88-b959-4272-a784-e6e96ab10098\") "
Sep 17 14:31:26.067432 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:26.067386    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7be60b88-b959-4272-a784-e6e96ab10098-istio-token\") pod \"7be60b88-b959-4272-a784-e6e96ab10098\" (UID: \"7be60b88-b959-4272-a784-e6e96ab10098\") "
Sep 17 14:31:26.067491 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:26.067441    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7be60b88-b959-4272-a784-e6e96ab10098-credential-socket\") pod \"7be60b88-b959-4272-a784-e6e96ab10098\" (UID: \"7be60b88-b959-4272-a784-e6e96ab10098\") "
Sep 17 14:31:26.067491 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:26.067464    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7be60b88-b959-4272-a784-e6e96ab10098-istio-envoy\") pod \"7be60b88-b959-4272-a784-e6e96ab10098\" (UID: \"7be60b88-b959-4272-a784-e6e96ab10098\") "
Sep 17 14:31:26.067599 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:26.067473    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7be60b88-b959-4272-a784-e6e96ab10098-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7be60b88-b959-4272-a784-e6e96ab10098" (UID: "7be60b88-b959-4272-a784-e6e96ab10098"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:31:26.067599 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:26.067548    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7be60b88-b959-4272-a784-e6e96ab10098-istio-data\") pod \"7be60b88-b959-4272-a784-e6e96ab10098\" (UID: \"7be60b88-b959-4272-a784-e6e96ab10098\") "
Sep 17 14:31:26.067599 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:26.067589    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dckps\" (UniqueName: \"kubernetes.io/projected/7be60b88-b959-4272-a784-e6e96ab10098-kube-api-access-dckps\") pod \"7be60b88-b959-4272-a784-e6e96ab10098\" (UID: \"7be60b88-b959-4272-a784-e6e96ab10098\") "
Sep 17 14:31:26.067599 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:26.067623    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7be60b88-b959-4272-a784-e6e96ab10098-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7be60b88-b959-4272-a784-e6e96ab10098" (UID: "7be60b88-b959-4272-a784-e6e96ab10098"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:31:26.067881 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:26.067662    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7be60b88-b959-4272-a784-e6e96ab10098-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7be60b88-b959-4272-a784-e6e96ab10098" (UID: "7be60b88-b959-4272-a784-e6e96ab10098"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:31:26.067881 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:26.067758    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7be60b88-b959-4272-a784-e6e96ab10098-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7be60b88-b959-4272-a784-e6e96ab10098" (UID: "7be60b88-b959-4272-a784-e6e96ab10098"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:31:26.067881 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:26.067863    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7be60b88-b959-4272-a784-e6e96ab10098-istio-data" (OuterVolumeSpecName: "istio-data") pod "7be60b88-b959-4272-a784-e6e96ab10098" (UID: "7be60b88-b959-4272-a784-e6e96ab10098"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:31:26.068142 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:26.068123    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7be60b88-b959-4272-a784-e6e96ab10098-istio-data\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:31:26.068222 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:26.068146    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7be60b88-b959-4272-a784-e6e96ab10098-workload-certs\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:31:26.068222 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:26.068160    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7be60b88-b959-4272-a784-e6e96ab10098-istiod-ca-cert\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:31:26.068222 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:26.068172    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7be60b88-b959-4272-a784-e6e96ab10098-workload-socket\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:31:26.068222 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:26.068184    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7be60b88-b959-4272-a784-e6e96ab10098-credential-socket\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:31:26.069438 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:26.069415    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7be60b88-b959-4272-a784-e6e96ab10098-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7be60b88-b959-4272-a784-e6e96ab10098" (UID: "7be60b88-b959-4272-a784-e6e96ab10098"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 14:31:26.069798 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:26.069777    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7be60b88-b959-4272-a784-e6e96ab10098-kube-api-access-dckps" (OuterVolumeSpecName: "kube-api-access-dckps") pod "7be60b88-b959-4272-a784-e6e96ab10098" (UID: "7be60b88-b959-4272-a784-e6e96ab10098"). InnerVolumeSpecName "kube-api-access-dckps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:31:26.069962 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:26.069942    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7be60b88-b959-4272-a784-e6e96ab10098-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7be60b88-b959-4272-a784-e6e96ab10098" (UID: "7be60b88-b959-4272-a784-e6e96ab10098"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:31:26.069962 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:26.069950    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7be60b88-b959-4272-a784-e6e96ab10098-istio-token" (OuterVolumeSpecName: "istio-token") pod "7be60b88-b959-4272-a784-e6e96ab10098" (UID: "7be60b88-b959-4272-a784-e6e96ab10098"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:31:26.169577 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:26.169542    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7be60b88-b959-4272-a784-e6e96ab10098-istio-envoy\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:31:26.169577 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:26.169573    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dckps\" (UniqueName: \"kubernetes.io/projected/7be60b88-b959-4272-a784-e6e96ab10098-kube-api-access-dckps\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:31:26.169577 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:26.169584    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7be60b88-b959-4272-a784-e6e96ab10098-istio-podinfo\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:31:26.169848 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:26.169594    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7be60b88-b959-4272-a784-e6e96ab10098-istio-token\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:31:26.364911 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:26.364876    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv"]
Sep 17 14:31:26.368991 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:26.368964    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-a954c8-openshift-default-6d4655bc96-rkjgv"]
Sep 17 14:31:26.457243 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:26.457206    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:27.260786 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:27.260747    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:27.497800 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:27.497766    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7be60b88-b959-4272-a784-e6e96ab10098" path="/var/lib/kubelet/pods/7be60b88-b959-4272-a784-e6e96ab10098/volumes"
Sep 17 14:31:27.928339 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:27.928295    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:31.177845 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:31.177808    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:36.764744 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:36.764709    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:36.894538 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:36.894491    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cf94fd-mqcp9"]
Sep 17 14:31:36.897075 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:36.897050    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cf94fd-mqcp9"]
Sep 17 14:31:36.920436 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:36.920359    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:37.115420 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:37.115379    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:37.337103 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:37.337069    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:37.494681 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:37.494649    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9963b26f-c2cb-4019-814f-869e4e53406c" path="/var/lib/kubelet/pods/9963b26f-c2cb-4019-814f-869e4e53406c/volumes"
Sep 17 14:31:37.556619 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:37.556571    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:38.242988 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:38.242954    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:39.029999 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:39.029956    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:39.785433 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:39.785404    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:40.535110 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:40.535080    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:40.633772 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:40.633735    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-1269b2-2s8s8"]
Sep 17 14:31:40.634229 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:40.634215    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7be60b88-b959-4272-a784-e6e96ab10098" containerName="istio-proxy"
Sep 17 14:31:40.634279 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:40.634230    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7be60b88-b959-4272-a784-e6e96ab10098" containerName="istio-proxy"
Sep 17 14:31:40.634279 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:40.634240    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7be60b88-b959-4272-a784-e6e96ab10098" containerName="istio-proxy"
Sep 17 14:31:40.634279 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:40.634247    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7be60b88-b959-4272-a784-e6e96ab10098" containerName="istio-proxy"
Sep 17 14:31:40.634279 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:40.634261    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9963b26f-c2cb-4019-814f-869e4e53406c" containerName="revoke-keys"
Sep 17 14:31:40.634279 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:40.634266    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9963b26f-c2cb-4019-814f-869e4e53406c" containerName="revoke-keys"
Sep 17 14:31:40.634436 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:40.634348    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7be60b88-b959-4272-a784-e6e96ab10098" containerName="istio-proxy"
Sep 17 14:31:40.634436 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:40.634357    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7be60b88-b959-4272-a784-e6e96ab10098" containerName="istio-proxy"
Sep 17 14:31:40.634436 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:40.634363    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9963b26f-c2cb-4019-814f-869e4e53406c" containerName="revoke-keys"
Sep 17 14:31:40.636463 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:40.636448    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-1269b2-2s8s8"
Sep 17 14:31:40.638884 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:40.638862    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ddwrn\""
Sep 17 14:31:40.649164 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:40.649147    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-1269b2-2s8s8"]
Sep 17 14:31:40.686765 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:40.686738    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:40.732223 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:40.732190    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64d6c82d-8e0b-455e-884a-619f5663e660-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-1269b2-2s8s8\" (UID: \"64d6c82d-8e0b-455e-884a-619f5663e660\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-1269b2-2s8s8"
Sep 17 14:31:40.833590 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:40.833499    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64d6c82d-8e0b-455e-884a-619f5663e660-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-1269b2-2s8s8\" (UID: \"64d6c82d-8e0b-455e-884a-619f5663e660\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-1269b2-2s8s8"
Sep 17 14:31:40.834179 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:40.834146    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64d6c82d-8e0b-455e-884a-619f5663e660-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-1269b2-2s8s8\" (UID: \"64d6c82d-8e0b-455e-884a-619f5663e660\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-1269b2-2s8s8"
Sep 17 14:31:40.947205 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:40.947168    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-1269b2-2s8s8"
Sep 17 14:31:41.226967 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:41.226934    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-1269b2-2s8s8"]
Sep 17 14:31:41.228234 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:31:41.228210    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64d6c82d_8e0b_455e_884a_619f5663e660.slice/crio-8345e67bdb920e791778eb83fe002c091da89f170f23bbb51e7539e884c04846 WatchSource:0}: Error finding container 8345e67bdb920e791778eb83fe002c091da89f170f23bbb51e7539e884c04846: Status 404 returned error can't find the container with id 8345e67bdb920e791778eb83fe002c091da89f170f23bbb51e7539e884c04846
Sep 17 14:31:41.489664 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:41.489550    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:42.013989 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:42.013953    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:42.156357 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:42.156317    2582 generic.go:356] "Generic (PLEG): container finished" podID="64d6c82d-8e0b-455e-884a-619f5663e660" containerID="e5ef38dd92a69a63826f54271d93d0bf195620a93a5431c1aa89b9844a8afea9" exitCode=0
Sep 17 14:31:42.156539 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:42.156377    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-1269b2-2s8s8" event={"ID":"64d6c82d-8e0b-455e-884a-619f5663e660","Type":"ContainerDied","Data":"e5ef38dd92a69a63826f54271d93d0bf195620a93a5431c1aa89b9844a8afea9"}
Sep 17 14:31:42.156539 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:42.156443    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-1269b2-2s8s8" event={"ID":"64d6c82d-8e0b-455e-884a-619f5663e660","Type":"ContainerStarted","Data":"8345e67bdb920e791778eb83fe002c091da89f170f23bbb51e7539e884c04846"}
Sep 17 14:31:42.734367 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:42.734328    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:43.308482 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:43.308457    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-1269b2-2s8s8"
Sep 17 14:31:43.469845 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:43.469813    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64d6c82d-8e0b-455e-884a-619f5663e660-maas-api-service-ca\") pod \"64d6c82d-8e0b-455e-884a-619f5663e660\" (UID: \"64d6c82d-8e0b-455e-884a-619f5663e660\") "
Sep 17 14:31:43.470186 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:43.470164    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64d6c82d-8e0b-455e-884a-619f5663e660-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "64d6c82d-8e0b-455e-884a-619f5663e660" (UID: "64d6c82d-8e0b-455e-884a-619f5663e660"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:31:43.551777 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:43.551742    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:43.571181 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:43.571148    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64d6c82d-8e0b-455e-884a-619f5663e660-maas-api-service-ca\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:31:43.808352 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:43.808264    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:43.910599 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:43.910557    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:43.986039 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:43.986006    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb"]
Sep 17 14:31:43.986258 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:43.986230    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb" podUID="225ff0fb-8843-44e7-a8c1-67a86eebcf25" containerName="istio-proxy" containerID="cri-o://8aeaca45a8f40044d7ba9a9285231243daf5b4b1aa1a2e50531c460cd0ea5cd4" gracePeriod=30
Sep 17 14:31:44.033103 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:44.033063    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:44.170392 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:44.170306    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-1269b2-2s8s8" event={"ID":"64d6c82d-8e0b-455e-884a-619f5663e660","Type":"ContainerDied","Data":"8345e67bdb920e791778eb83fe002c091da89f170f23bbb51e7539e884c04846"}
Sep 17 14:31:44.170392 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:44.170330    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-1269b2-2s8s8"
Sep 17 14:31:44.170392 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:44.170354    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8345e67bdb920e791778eb83fe002c091da89f170f23bbb51e7539e884c04846"
Sep 17 14:31:44.359928 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:44.359891    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c98839-xt2ld"]
Sep 17 14:31:44.360375 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:44.360363    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64d6c82d-8e0b-455e-884a-619f5663e660" containerName="revoke-keys"
Sep 17 14:31:44.360375 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:44.360376    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="64d6c82d-8e0b-455e-884a-619f5663e660" containerName="revoke-keys"
Sep 17 14:31:44.360469 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:44.360460    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="64d6c82d-8e0b-455e-884a-619f5663e660" containerName="revoke-keys"
Sep 17 14:31:44.362719 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:44.362700    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c98839-xt2ld"
Sep 17 14:31:44.365405 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:44.365386    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ddwrn\""
Sep 17 14:31:44.381052 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:44.381029    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c98839-xt2ld"]
Sep 17 14:31:44.407695 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:44.402886    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:44.481822 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:44.481783    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb40d353-a95e-4462-b473-e46ecc3a2ef5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-c98839-xt2ld\" (UID: \"cb40d353-a95e-4462-b473-e46ecc3a2ef5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c98839-xt2ld"
Sep 17 14:31:44.582573 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:44.582533    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb40d353-a95e-4462-b473-e46ecc3a2ef5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-c98839-xt2ld\" (UID: \"cb40d353-a95e-4462-b473-e46ecc3a2ef5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c98839-xt2ld"
Sep 17 14:31:44.583375 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:44.583347    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb40d353-a95e-4462-b473-e46ecc3a2ef5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-c98839-xt2ld\" (UID: \"cb40d353-a95e-4462-b473-e46ecc3a2ef5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c98839-xt2ld"
Sep 17 14:31:44.673456 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:44.673416    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c98839-xt2ld"
Sep 17 14:31:44.952798 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:44.952771    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c98839-xt2ld"]
Sep 17 14:31:44.954725 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:31:44.954699    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb40d353_a95e_4462_b473_e46ecc3a2ef5.slice/crio-79644c5a44da94786e54b79cf6af259204aea7caa00c294b0157a92d663ab8dc WatchSource:0}: Error finding container 79644c5a44da94786e54b79cf6af259204aea7caa00c294b0157a92d663ab8dc: Status 404 returned error can't find the container with id 79644c5a44da94786e54b79cf6af259204aea7caa00c294b0157a92d663ab8dc
Sep 17 14:31:45.179002 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:45.178966    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c98839-xt2ld" event={"ID":"cb40d353-a95e-4462-b473-e46ecc3a2ef5","Type":"ContainerStarted","Data":"79644c5a44da94786e54b79cf6af259204aea7caa00c294b0157a92d663ab8dc"}
Sep 17 14:31:45.512407 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:45.512364    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-1269b2/embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt"]
Sep 17 14:31:45.556846 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:45.556792    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:45.672639 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:45.672595    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:46.186881 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:46.186845    2582 generic.go:356] "Generic (PLEG): container finished" podID="cb40d353-a95e-4462-b473-e46ecc3a2ef5" containerID="609b7b8a94a6a490036df07c976910830321dac4fd76e6fb31c4f7db16748d4d" exitCode=0
Sep 17 14:31:46.187075 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:46.186926    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c98839-xt2ld" event={"ID":"cb40d353-a95e-4462-b473-e46ecc3a2ef5","Type":"ContainerDied","Data":"609b7b8a94a6a490036df07c976910830321dac4fd76e6fb31c4f7db16748d4d"}
Sep 17 14:31:46.449726 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:46.449697    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:47.358477 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:47.358451    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c98839-xt2ld"
Sep 17 14:31:47.511165 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:47.511080    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb40d353-a95e-4462-b473-e46ecc3a2ef5-maas-api-service-ca\") pod \"cb40d353-a95e-4462-b473-e46ecc3a2ef5\" (UID: \"cb40d353-a95e-4462-b473-e46ecc3a2ef5\") "
Sep 17 14:31:47.511451 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:47.511427    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb40d353-a95e-4462-b473-e46ecc3a2ef5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cb40d353-a95e-4462-b473-e46ecc3a2ef5" (UID: "cb40d353-a95e-4462-b473-e46ecc3a2ef5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:31:47.561927 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:47.561884    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:47.612871 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:47.612841    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb40d353-a95e-4462-b473-e46ecc3a2ef5-maas-api-service-ca\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:31:48.202020 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:48.201992    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c98839-xt2ld"
Sep 17 14:31:48.202020 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:48.202004    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c98839-xt2ld" event={"ID":"cb40d353-a95e-4462-b473-e46ecc3a2ef5","Type":"ContainerDied","Data":"79644c5a44da94786e54b79cf6af259204aea7caa00c294b0157a92d663ab8dc"}
Sep 17 14:31:48.202283 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:48.202074    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="79644c5a44da94786e54b79cf6af259204aea7caa00c294b0157a92d663ab8dc"
Sep 17 14:31:48.281775 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:48.281731    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:49.171342 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:49.171307    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:49.212943 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:49.212910    2582 generic.go:356] "Generic (PLEG): container finished" podID="225ff0fb-8843-44e7-a8c1-67a86eebcf25" containerID="8aeaca45a8f40044d7ba9a9285231243daf5b4b1aa1a2e50531c460cd0ea5cd4" exitCode=0
Sep 17 14:31:49.213106 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:49.212958    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb" event={"ID":"225ff0fb-8843-44e7-a8c1-67a86eebcf25","Type":"ContainerDied","Data":"8aeaca45a8f40044d7ba9a9285231243daf5b4b1aa1a2e50531c460cd0ea5cd4"}
Sep 17 14:31:49.213106 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:49.213030    2582 scope.go:117] "RemoveContainer" containerID="f6a0fcc25a0b82a2bd124bbed88079b8076d60f8ba56c058f0ba955dc43456ae"
Sep 17 14:31:49.263842 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:49.263784    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb"
Sep 17 14:31:49.432100 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:49.432048    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/225ff0fb-8843-44e7-a8c1-67a86eebcf25-workload-socket\") pod \"225ff0fb-8843-44e7-a8c1-67a86eebcf25\" (UID: \"225ff0fb-8843-44e7-a8c1-67a86eebcf25\") "
Sep 17 14:31:49.432309 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:49.432136    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/225ff0fb-8843-44e7-a8c1-67a86eebcf25-istio-data\") pod \"225ff0fb-8843-44e7-a8c1-67a86eebcf25\" (UID: \"225ff0fb-8843-44e7-a8c1-67a86eebcf25\") "
Sep 17 14:31:49.432309 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:49.432226    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/225ff0fb-8843-44e7-a8c1-67a86eebcf25-credential-socket\") pod \"225ff0fb-8843-44e7-a8c1-67a86eebcf25\" (UID: \"225ff0fb-8843-44e7-a8c1-67a86eebcf25\") "
Sep 17 14:31:49.432309 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:49.432279    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/225ff0fb-8843-44e7-a8c1-67a86eebcf25-istio-podinfo\") pod \"225ff0fb-8843-44e7-a8c1-67a86eebcf25\" (UID: \"225ff0fb-8843-44e7-a8c1-67a86eebcf25\") "
Sep 17 14:31:49.432483 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:49.432312    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/225ff0fb-8843-44e7-a8c1-67a86eebcf25-workload-certs\") pod \"225ff0fb-8843-44e7-a8c1-67a86eebcf25\" (UID: \"225ff0fb-8843-44e7-a8c1-67a86eebcf25\") "
Sep 17 14:31:49.432483 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:49.432341    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/225ff0fb-8843-44e7-a8c1-67a86eebcf25-istio-envoy\") pod \"225ff0fb-8843-44e7-a8c1-67a86eebcf25\" (UID: \"225ff0fb-8843-44e7-a8c1-67a86eebcf25\") "
Sep 17 14:31:49.432483 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:49.432381    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/225ff0fb-8843-44e7-a8c1-67a86eebcf25-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "225ff0fb-8843-44e7-a8c1-67a86eebcf25" (UID: "225ff0fb-8843-44e7-a8c1-67a86eebcf25"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:31:49.432483 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:49.432390    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/225ff0fb-8843-44e7-a8c1-67a86eebcf25-istiod-ca-cert\") pod \"225ff0fb-8843-44e7-a8c1-67a86eebcf25\" (UID: \"225ff0fb-8843-44e7-a8c1-67a86eebcf25\") "
Sep 17 14:31:49.432483 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:49.432413    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/225ff0fb-8843-44e7-a8c1-67a86eebcf25-istio-data" (OuterVolumeSpecName: "istio-data") pod "225ff0fb-8843-44e7-a8c1-67a86eebcf25" (UID: "225ff0fb-8843-44e7-a8c1-67a86eebcf25"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:31:49.432771 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:49.432512    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/225ff0fb-8843-44e7-a8c1-67a86eebcf25-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "225ff0fb-8843-44e7-a8c1-67a86eebcf25" (UID: "225ff0fb-8843-44e7-a8c1-67a86eebcf25"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:31:49.432771 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:49.432516    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5sfhv\" (UniqueName: \"kubernetes.io/projected/225ff0fb-8843-44e7-a8c1-67a86eebcf25-kube-api-access-5sfhv\") pod \"225ff0fb-8843-44e7-a8c1-67a86eebcf25\" (UID: \"225ff0fb-8843-44e7-a8c1-67a86eebcf25\") "
Sep 17 14:31:49.432771 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:49.432593    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/225ff0fb-8843-44e7-a8c1-67a86eebcf25-istio-token\") pod \"225ff0fb-8843-44e7-a8c1-67a86eebcf25\" (UID: \"225ff0fb-8843-44e7-a8c1-67a86eebcf25\") "
Sep 17 14:31:49.432771 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:49.432676    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/225ff0fb-8843-44e7-a8c1-67a86eebcf25-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "225ff0fb-8843-44e7-a8c1-67a86eebcf25" (UID: "225ff0fb-8843-44e7-a8c1-67a86eebcf25"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:31:49.432771 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:49.432736    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/225ff0fb-8843-44e7-a8c1-67a86eebcf25-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "225ff0fb-8843-44e7-a8c1-67a86eebcf25" (UID: "225ff0fb-8843-44e7-a8c1-67a86eebcf25"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:31:49.433098 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:49.433072    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/225ff0fb-8843-44e7-a8c1-67a86eebcf25-workload-socket\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:31:49.433098 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:49.433100    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/225ff0fb-8843-44e7-a8c1-67a86eebcf25-istio-data\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:31:49.433282 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:49.433116    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/225ff0fb-8843-44e7-a8c1-67a86eebcf25-credential-socket\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:31:49.433282 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:49.433129    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/225ff0fb-8843-44e7-a8c1-67a86eebcf25-workload-certs\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:31:49.433282 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:49.433143    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/225ff0fb-8843-44e7-a8c1-67a86eebcf25-istiod-ca-cert\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:31:49.434701 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:49.434678    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/225ff0fb-8843-44e7-a8c1-67a86eebcf25-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "225ff0fb-8843-44e7-a8c1-67a86eebcf25" (UID: "225ff0fb-8843-44e7-a8c1-67a86eebcf25"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 14:31:49.434970 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:49.434953    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/225ff0fb-8843-44e7-a8c1-67a86eebcf25-istio-token" (OuterVolumeSpecName: "istio-token") pod "225ff0fb-8843-44e7-a8c1-67a86eebcf25" (UID: "225ff0fb-8843-44e7-a8c1-67a86eebcf25"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:31:49.435162 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:49.435147    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/225ff0fb-8843-44e7-a8c1-67a86eebcf25-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "225ff0fb-8843-44e7-a8c1-67a86eebcf25" (UID: "225ff0fb-8843-44e7-a8c1-67a86eebcf25"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:31:49.435333 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:49.435310    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/225ff0fb-8843-44e7-a8c1-67a86eebcf25-kube-api-access-5sfhv" (OuterVolumeSpecName: "kube-api-access-5sfhv") pod "225ff0fb-8843-44e7-a8c1-67a86eebcf25" (UID: "225ff0fb-8843-44e7-a8c1-67a86eebcf25"). InnerVolumeSpecName "kube-api-access-5sfhv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:31:49.534995 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:49.534907    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/225ff0fb-8843-44e7-a8c1-67a86eebcf25-istio-podinfo\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:31:49.534995 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:49.534946    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/225ff0fb-8843-44e7-a8c1-67a86eebcf25-istio-envoy\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:31:49.534995 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:49.534987    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5sfhv\" (UniqueName: \"kubernetes.io/projected/225ff0fb-8843-44e7-a8c1-67a86eebcf25-kube-api-access-5sfhv\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:31:49.535196 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:49.535002    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/225ff0fb-8843-44e7-a8c1-67a86eebcf25-istio-token\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:31:49.799234 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:49.799124    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:50.220584 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:50.220552    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb" event={"ID":"225ff0fb-8843-44e7-a8c1-67a86eebcf25","Type":"ContainerDied","Data":"734a2201d1f3494b69ec9871f5821e6d1cb73f03ae3055c544916aa346a0d96f"}
Sep 17 14:31:50.220978 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:50.220595    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb"
Sep 17 14:31:50.220978 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:50.220662    2582 scope.go:117] "RemoveContainer" containerID="8aeaca45a8f40044d7ba9a9285231243daf5b4b1aa1a2e50531c460cd0ea5cd4"
Sep 17 14:31:50.240540 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:50.240512    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb"]
Sep 17 14:31:50.244847 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:50.244820    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-cf94fd-openshift-default-545595cdc4-j9vhb"]
Sep 17 14:31:50.572965 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:50.572878    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:51.495682 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:51.495645    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="225ff0fb-8843-44e7-a8c1-67a86eebcf25" path="/var/lib/kubelet/pods/225ff0fb-8843-44e7-a8c1-67a86eebcf25/volumes"
Sep 17 14:31:55.055324 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:55.055288    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:55.195258 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:55.195221    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:55.234960 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:55.234931    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c98839-xt2ld"]
Sep 17 14:31:55.237399 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:55.237375    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c98839-xt2ld"]
Sep 17 14:31:55.329850 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:55.329761    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:55.499992 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:55.497302    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb40d353-a95e-4462-b473-e46ecc3a2ef5" path="/var/lib/kubelet/pods/cb40d353-a95e-4462-b473-e46ecc3a2ef5/volumes"
Sep 17 14:31:55.666457 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:55.666360    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:55.932360 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:55.932277    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:56.423594 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:56.423550    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-1269b2-29827590-rph8m"]
Sep 17 14:31:56.428573 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:56.428538    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-1269b2-29827590-rph8m"]
Sep 17 14:31:56.456765 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:56.456730    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-1269b2-fdd4895d7-mr8fh"]
Sep 17 14:31:56.457026 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:56.456996    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-1269b2-fdd4895d7-mr8fh" podUID="a17e2253-de02-4951-b5f4-e6eff7611b15" containerName="payload-processing" containerID="cri-o://efae5a15053b5d195b0b07441717f32c9cb471089b38575c9ce5b2856db67071" gracePeriod=30
Sep 17 14:31:56.518527 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:56.518476    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:56.673443 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:56.673409    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-1269b2-2s8s8"]
Sep 17 14:31:56.678048 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:56.677979    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-1269b2-2s8s8"]
Sep 17 14:31:56.708373 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:56.708337    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:56.718305 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:56.718284    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-1269b2-fdd4895d7-mr8fh"
Sep 17 14:31:56.817312 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:56.817281    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d8pc4\" (UniqueName: \"kubernetes.io/projected/a17e2253-de02-4951-b5f4-e6eff7611b15-kube-api-access-d8pc4\") pod \"a17e2253-de02-4951-b5f4-e6eff7611b15\" (UID: \"a17e2253-de02-4951-b5f4-e6eff7611b15\") "
Sep 17 14:31:56.817485 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:56.817380    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a17e2253-de02-4951-b5f4-e6eff7611b15-plugins-config-volume\") pod \"a17e2253-de02-4951-b5f4-e6eff7611b15\" (UID: \"a17e2253-de02-4951-b5f4-e6eff7611b15\") "
Sep 17 14:31:56.817767 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:56.817741    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a17e2253-de02-4951-b5f4-e6eff7611b15-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a17e2253-de02-4951-b5f4-e6eff7611b15" (UID: "a17e2253-de02-4951-b5f4-e6eff7611b15"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:31:56.817847 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:56.817836    2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a17e2253-de02-4951-b5f4-e6eff7611b15-plugins-config-volume\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:31:56.819471 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:56.819446    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a17e2253-de02-4951-b5f4-e6eff7611b15-kube-api-access-d8pc4" (OuterVolumeSpecName: "kube-api-access-d8pc4") pod "a17e2253-de02-4951-b5f4-e6eff7611b15" (UID: "a17e2253-de02-4951-b5f4-e6eff7611b15"). InnerVolumeSpecName "kube-api-access-d8pc4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:31:56.918317 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:56.918277    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:56.918555 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:56.918530    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8pc4\" (UniqueName: \"kubernetes.io/projected/a17e2253-de02-4951-b5f4-e6eff7611b15-kube-api-access-d8pc4\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:31:57.071980 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:57.071932    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:57.282897 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:57.282864    2582 generic.go:356] "Generic (PLEG): container finished" podID="a17e2253-de02-4951-b5f4-e6eff7611b15" containerID="efae5a15053b5d195b0b07441717f32c9cb471089b38575c9ce5b2856db67071" exitCode=0
Sep 17 14:31:57.283071 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:57.282910    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-1269b2-fdd4895d7-mr8fh"
Sep 17 14:31:57.283071 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:57.282952    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-1269b2-fdd4895d7-mr8fh" event={"ID":"a17e2253-de02-4951-b5f4-e6eff7611b15","Type":"ContainerDied","Data":"efae5a15053b5d195b0b07441717f32c9cb471089b38575c9ce5b2856db67071"}
Sep 17 14:31:57.283071 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:57.283007    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-1269b2-fdd4895d7-mr8fh" event={"ID":"a17e2253-de02-4951-b5f4-e6eff7611b15","Type":"ContainerDied","Data":"72d1cf01228dd39bf520fa45f970aec7e3f0e70e83d642e536a66b799c1511bf"}
Sep 17 14:31:57.283071 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:57.283035    2582 scope.go:117] "RemoveContainer" containerID="efae5a15053b5d195b0b07441717f32c9cb471089b38575c9ce5b2856db67071"
Sep 17 14:31:57.293425 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:57.293407    2582 scope.go:117] "RemoveContainer" containerID="efae5a15053b5d195b0b07441717f32c9cb471089b38575c9ce5b2856db67071"
Sep 17 14:31:57.293780 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:31:57.293734    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efae5a15053b5d195b0b07441717f32c9cb471089b38575c9ce5b2856db67071\": container with ID starting with efae5a15053b5d195b0b07441717f32c9cb471089b38575c9ce5b2856db67071 not found: ID does not exist" containerID="efae5a15053b5d195b0b07441717f32c9cb471089b38575c9ce5b2856db67071"
Sep 17 14:31:57.293876 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:57.293781    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efae5a15053b5d195b0b07441717f32c9cb471089b38575c9ce5b2856db67071"} err="failed to get container status \"efae5a15053b5d195b0b07441717f32c9cb471089b38575c9ce5b2856db67071\": rpc error: code = NotFound desc = could not find container \"efae5a15053b5d195b0b07441717f32c9cb471089b38575c9ce5b2856db67071\": container with ID starting with efae5a15053b5d195b0b07441717f32c9cb471089b38575c9ce5b2856db67071 not found: ID does not exist"
Sep 17 14:31:57.313264 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:57.313224    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-1269b2-fdd4895d7-mr8fh"]
Sep 17 14:31:57.333261 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:57.333173    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-1269b2-fdd4895d7-mr8fh"]
Sep 17 14:31:57.494948 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:57.494910    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64d6c82d-8e0b-455e-884a-619f5663e660" path="/var/lib/kubelet/pods/64d6c82d-8e0b-455e-884a-619f5663e660/volumes"
Sep 17 14:31:57.495521 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:57.495498    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a17e2253-de02-4951-b5f4-e6eff7611b15" path="/var/lib/kubelet/pods/a17e2253-de02-4951-b5f4-e6eff7611b15/volumes"
Sep 17 14:31:57.495884 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:57.495864    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9a93782-7587-44b5-a57b-1e6522631a2a" path="/var/lib/kubelet/pods/c9a93782-7587-44b5-a57b-1e6522631a2a/volumes"
Sep 17 14:31:57.776395 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:57.776359    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:58.561334 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:58.561307    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:59.330923 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:59.330887    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:31:59.784187 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:31:59.784154    2582 scope.go:117] "RemoveContainer" containerID="ca986e3a65b2ef269b6dc064ee37ab78bbb389eacd8c66ecf75e127ec9e74ad7"
Sep 17 14:32:00.086953 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:00.086868    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:32:00.533695 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:00.533656    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-1269b2/embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt" podUID="31b949bb-4068-49d8-9df6-be2801c9eef9" containerName="main" containerID="cri-o://d70248c1e082876843e90b7d2b35e8ce1b983807e0e51abf38275d3a065cd3b0" gracePeriod=45
Sep 17 14:32:00.802782 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:00.802755    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-1269b2/embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt"
Sep 17 14:32:00.861998 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:00.861963    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31b949bb-4068-49d8-9df6-be2801c9eef9-tmp-dir\") pod \"31b949bb-4068-49d8-9df6-be2801c9eef9\" (UID: \"31b949bb-4068-49d8-9df6-be2801c9eef9\") "
Sep 17 14:32:00.862178 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:00.862020    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31b949bb-4068-49d8-9df6-be2801c9eef9-model-cache\") pod \"31b949bb-4068-49d8-9df6-be2801c9eef9\" (UID: \"31b949bb-4068-49d8-9df6-be2801c9eef9\") "
Sep 17 14:32:00.862178 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:00.862095    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31b949bb-4068-49d8-9df6-be2801c9eef9-tls-certs\") pod \"31b949bb-4068-49d8-9df6-be2801c9eef9\" (UID: \"31b949bb-4068-49d8-9df6-be2801c9eef9\") "
Sep 17 14:32:00.862178 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:00.862166    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31b949bb-4068-49d8-9df6-be2801c9eef9-dshm\") pod \"31b949bb-4068-49d8-9df6-be2801c9eef9\" (UID: \"31b949bb-4068-49d8-9df6-be2801c9eef9\") "
Sep 17 14:32:00.862369 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:00.862217    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31b949bb-4068-49d8-9df6-be2801c9eef9-home\") pod \"31b949bb-4068-49d8-9df6-be2801c9eef9\" (UID: \"31b949bb-4068-49d8-9df6-be2801c9eef9\") "
Sep 17 14:32:00.862369 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:00.862274    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31b949bb-4068-49d8-9df6-be2801c9eef9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "31b949bb-4068-49d8-9df6-be2801c9eef9" (UID: "31b949bb-4068-49d8-9df6-be2801c9eef9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:32:00.862369 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:00.862289    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wwkpj\" (UniqueName: \"kubernetes.io/projected/31b949bb-4068-49d8-9df6-be2801c9eef9-kube-api-access-wwkpj\") pod \"31b949bb-4068-49d8-9df6-be2801c9eef9\" (UID: \"31b949bb-4068-49d8-9df6-be2801c9eef9\") "
Sep 17 14:32:00.862369 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:00.862315    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31b949bb-4068-49d8-9df6-be2801c9eef9-model-cache" (OuterVolumeSpecName: "model-cache") pod "31b949bb-4068-49d8-9df6-be2801c9eef9" (UID: "31b949bb-4068-49d8-9df6-be2801c9eef9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:32:00.862723 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:00.862598    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31b949bb-4068-49d8-9df6-be2801c9eef9-home" (OuterVolumeSpecName: "home") pod "31b949bb-4068-49d8-9df6-be2801c9eef9" (UID: "31b949bb-4068-49d8-9df6-be2801c9eef9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:32:00.862871 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:00.862766    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31b949bb-4068-49d8-9df6-be2801c9eef9-home\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:32:00.862871 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:00.862788    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31b949bb-4068-49d8-9df6-be2801c9eef9-tmp-dir\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:32:00.862871 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:00.862802    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31b949bb-4068-49d8-9df6-be2801c9eef9-model-cache\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:32:00.864738 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:00.864703    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31b949bb-4068-49d8-9df6-be2801c9eef9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "31b949bb-4068-49d8-9df6-be2801c9eef9" (UID: "31b949bb-4068-49d8-9df6-be2801c9eef9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:32:00.864906 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:00.864883    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31b949bb-4068-49d8-9df6-be2801c9eef9-dshm" (OuterVolumeSpecName: "dshm") pod "31b949bb-4068-49d8-9df6-be2801c9eef9" (UID: "31b949bb-4068-49d8-9df6-be2801c9eef9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:32:00.864973 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:00.864910    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31b949bb-4068-49d8-9df6-be2801c9eef9-kube-api-access-wwkpj" (OuterVolumeSpecName: "kube-api-access-wwkpj") pod "31b949bb-4068-49d8-9df6-be2801c9eef9" (UID: "31b949bb-4068-49d8-9df6-be2801c9eef9"). InnerVolumeSpecName "kube-api-access-wwkpj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:32:00.877074 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:00.877046    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:32:00.963408 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:00.963369    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wwkpj\" (UniqueName: \"kubernetes.io/projected/31b949bb-4068-49d8-9df6-be2801c9eef9-kube-api-access-wwkpj\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:32:00.963408 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:00.963399    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31b949bb-4068-49d8-9df6-be2801c9eef9-tls-certs\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:32:00.963408 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:00.963411    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31b949bb-4068-49d8-9df6-be2801c9eef9-dshm\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:32:01.310689 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:01.310646    2582 generic.go:356] "Generic (PLEG): container finished" podID="31b949bb-4068-49d8-9df6-be2801c9eef9" containerID="d70248c1e082876843e90b7d2b35e8ce1b983807e0e51abf38275d3a065cd3b0" exitCode=0
Sep 17 14:32:01.310841 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:01.310700    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-1269b2/embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt" event={"ID":"31b949bb-4068-49d8-9df6-be2801c9eef9","Type":"ContainerDied","Data":"d70248c1e082876843e90b7d2b35e8ce1b983807e0e51abf38275d3a065cd3b0"}
Sep 17 14:32:01.310841 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:01.310715    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-1269b2/embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt"
Sep 17 14:32:01.310841 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:01.310746    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-1269b2/embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt" event={"ID":"31b949bb-4068-49d8-9df6-be2801c9eef9","Type":"ContainerDied","Data":"9295d3e853bdeece55fa525bc975bdbdd24ce81623559c59c762063bda96cd4d"}
Sep 17 14:32:01.310841 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:01.310765    2582 scope.go:117] "RemoveContainer" containerID="d70248c1e082876843e90b7d2b35e8ce1b983807e0e51abf38275d3a065cd3b0"
Sep 17 14:32:01.324556 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:01.324534    2582 scope.go:117] "RemoveContainer" containerID="d70248c1e082876843e90b7d2b35e8ce1b983807e0e51abf38275d3a065cd3b0"
Sep 17 14:32:01.324879 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:32:01.324847    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d70248c1e082876843e90b7d2b35e8ce1b983807e0e51abf38275d3a065cd3b0\": container with ID starting with d70248c1e082876843e90b7d2b35e8ce1b983807e0e51abf38275d3a065cd3b0 not found: ID does not exist" containerID="d70248c1e082876843e90b7d2b35e8ce1b983807e0e51abf38275d3a065cd3b0"
Sep 17 14:32:01.324982 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:01.324878    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d70248c1e082876843e90b7d2b35e8ce1b983807e0e51abf38275d3a065cd3b0"} err="failed to get container status \"d70248c1e082876843e90b7d2b35e8ce1b983807e0e51abf38275d3a065cd3b0\": rpc error: code = NotFound desc = could not find container \"d70248c1e082876843e90b7d2b35e8ce1b983807e0e51abf38275d3a065cd3b0\": container with ID starting with d70248c1e082876843e90b7d2b35e8ce1b983807e0e51abf38275d3a065cd3b0 not found: ID does not exist"
Sep 17 14:32:01.336233 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:01.336204    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-1269b2/embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt"]
Sep 17 14:32:01.338627 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:01.338592    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-1269b2/embedding-w0-1269b2-kserve-d64bf9cd6-lj2vt"]
Sep 17 14:32:01.494871 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:01.494838    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31b949bb-4068-49d8-9df6-be2801c9eef9" path="/var/lib/kubelet/pods/31b949bb-4068-49d8-9df6-be2801c9eef9/volumes"
Sep 17 14:32:01.607022 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:01.606942    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:32:01.866626 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:01.866516    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:32:01.963017 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:01.962985    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:32:02.232624 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:02.232579    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b83f70-75h8b"]
Sep 17 14:32:02.233120 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:02.233102    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31b949bb-4068-49d8-9df6-be2801c9eef9" containerName="main"
Sep 17 14:32:02.233216 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:02.233122    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="31b949bb-4068-49d8-9df6-be2801c9eef9" containerName="main"
Sep 17 14:32:02.233216 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:02.233148    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a17e2253-de02-4951-b5f4-e6eff7611b15" containerName="payload-processing"
Sep 17 14:32:02.233216 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:02.233157    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a17e2253-de02-4951-b5f4-e6eff7611b15" containerName="payload-processing"
Sep 17 14:32:02.233216 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:02.233173    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="225ff0fb-8843-44e7-a8c1-67a86eebcf25" containerName="istio-proxy"
Sep 17 14:32:02.233216 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:02.233181    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="225ff0fb-8843-44e7-a8c1-67a86eebcf25" containerName="istio-proxy"
Sep 17 14:32:02.233216 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:02.233207    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="225ff0fb-8843-44e7-a8c1-67a86eebcf25" containerName="istio-proxy"
Sep 17 14:32:02.233216 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:02.233215    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="225ff0fb-8843-44e7-a8c1-67a86eebcf25" containerName="istio-proxy"
Sep 17 14:32:02.233550 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:02.233225    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb40d353-a95e-4462-b473-e46ecc3a2ef5" containerName="revoke-keys"
Sep 17 14:32:02.233550 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:02.233233    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb40d353-a95e-4462-b473-e46ecc3a2ef5" containerName="revoke-keys"
Sep 17 14:32:02.233550 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:02.233356    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="31b949bb-4068-49d8-9df6-be2801c9eef9" containerName="main"
Sep 17 14:32:02.233550 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:02.233371    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="225ff0fb-8843-44e7-a8c1-67a86eebcf25" containerName="istio-proxy"
Sep 17 14:32:02.233550 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:02.233385    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="225ff0fb-8843-44e7-a8c1-67a86eebcf25" containerName="istio-proxy"
Sep 17 14:32:02.233550 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:02.233394    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb40d353-a95e-4462-b473-e46ecc3a2ef5" containerName="revoke-keys"
Sep 17 14:32:02.233550 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:02.233406    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a17e2253-de02-4951-b5f4-e6eff7611b15" containerName="payload-processing"
Sep 17 14:32:02.236336 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:02.236313    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b83f70-75h8b"
Sep 17 14:32:02.238763 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:02.238742    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ddwrn\""
Sep 17 14:32:02.247971 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:02.247944    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b83f70-75h8b"]
Sep 17 14:32:02.276691 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:02.276658    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7daa0a3-8adf-4f0f-abdb-85d35779e02e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-b83f70-75h8b\" (UID: \"a7daa0a3-8adf-4f0f-abdb-85d35779e02e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b83f70-75h8b"
Sep 17 14:32:02.377699 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:02.377663    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7daa0a3-8adf-4f0f-abdb-85d35779e02e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-b83f70-75h8b\" (UID: \"a7daa0a3-8adf-4f0f-abdb-85d35779e02e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b83f70-75h8b"
Sep 17 14:32:02.378295 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:02.378275    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7daa0a3-8adf-4f0f-abdb-85d35779e02e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-b83f70-75h8b\" (UID: \"a7daa0a3-8adf-4f0f-abdb-85d35779e02e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b83f70-75h8b"
Sep 17 14:32:02.549768 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:02.549680    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b83f70-75h8b"
Sep 17 14:32:02.822061 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:02.822029    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b83f70-75h8b"]
Sep 17 14:32:02.823834 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:32:02.823807    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7daa0a3_8adf_4f0f_abdb_85d35779e02e.slice/crio-e64778242916cc314b46f83204a3f65c3b9939a57d105a58cee386bf817d0265 WatchSource:0}: Error finding container e64778242916cc314b46f83204a3f65c3b9939a57d105a58cee386bf817d0265: Status 404 returned error can't find the container with id e64778242916cc314b46f83204a3f65c3b9939a57d105a58cee386bf817d0265
Sep 17 14:32:03.020295 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:03.020260    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:32:03.169599 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:03.169538    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:32:03.326923 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:03.326884    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b83f70-75h8b" event={"ID":"a7daa0a3-8adf-4f0f-abdb-85d35779e02e","Type":"ContainerStarted","Data":"56c60a036bc64a6f336dc768825ddee926ff31b252513e31cc7f12d5178b56bc"}
Sep 17 14:32:03.327113 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:03.326949    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b83f70-75h8b" event={"ID":"a7daa0a3-8adf-4f0f-abdb-85d35779e02e","Type":"ContainerStarted","Data":"e64778242916cc314b46f83204a3f65c3b9939a57d105a58cee386bf817d0265"}
Sep 17 14:32:03.345032 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:03.344981    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b83f70-75h8b" podStartSLOduration=1.344965183 podStartE2EDuration="1.344965183s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:32:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:32:03.343347083 +0000 UTC m=+1624.344890557" watchObservedRunningTime="2026-09-17 14:32:03.344965183 +0000 UTC m=+1624.346508656"
Sep 17 14:32:03.361084 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:03.361046    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:32:03.430672 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:03.430557    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md"]
Sep 17 14:32:03.431235 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:03.431198    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md" podUID="68dd2b1c-ed84-4291-8fb7-f61779fd5b85" containerName="istio-proxy" containerID="cri-o://e1a29482fea6e73c670a158f9ff4259d158ff57276aa747806fb8ea0d330d790" gracePeriod=30
Sep 17 14:32:03.480662 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:03.480630    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:32:04.335318 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:04.335285    2582 generic.go:356] "Generic (PLEG): container finished" podID="a7daa0a3-8adf-4f0f-abdb-85d35779e02e" containerID="56c60a036bc64a6f336dc768825ddee926ff31b252513e31cc7f12d5178b56bc" exitCode=0
Sep 17 14:32:04.335853 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:04.335385    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b83f70-75h8b" event={"ID":"a7daa0a3-8adf-4f0f-abdb-85d35779e02e","Type":"ContainerDied","Data":"56c60a036bc64a6f336dc768825ddee926ff31b252513e31cc7f12d5178b56bc"}
Sep 17 14:32:05.505925 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:05.505890    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b83f70-75h8b"
Sep 17 14:32:05.610561 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:05.610465    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7daa0a3-8adf-4f0f-abdb-85d35779e02e-maas-api-service-ca\") pod \"a7daa0a3-8adf-4f0f-abdb-85d35779e02e\" (UID: \"a7daa0a3-8adf-4f0f-abdb-85d35779e02e\") "
Sep 17 14:32:05.611109 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:05.610913    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7daa0a3-8adf-4f0f-abdb-85d35779e02e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a7daa0a3-8adf-4f0f-abdb-85d35779e02e" (UID: "a7daa0a3-8adf-4f0f-abdb-85d35779e02e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:32:05.712523 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:05.712487    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7daa0a3-8adf-4f0f-abdb-85d35779e02e-maas-api-service-ca\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:32:06.350916 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:06.350889    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b83f70-75h8b"
Sep 17 14:32:06.351086 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:06.350884    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b83f70-75h8b" event={"ID":"a7daa0a3-8adf-4f0f-abdb-85d35779e02e","Type":"ContainerDied","Data":"e64778242916cc314b46f83204a3f65c3b9939a57d105a58cee386bf817d0265"}
Sep 17 14:32:06.351086 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:06.351019    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e64778242916cc314b46f83204a3f65c3b9939a57d105a58cee386bf817d0265"
Sep 17 14:32:08.710909 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:08.710838    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md"
Sep 17 14:32:08.848411 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:08.848380    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-istio-podinfo\") pod \"68dd2b1c-ed84-4291-8fb7-f61779fd5b85\" (UID: \"68dd2b1c-ed84-4291-8fb7-f61779fd5b85\") "
Sep 17 14:32:08.848640 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:08.848457    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-istio-envoy\") pod \"68dd2b1c-ed84-4291-8fb7-f61779fd5b85\" (UID: \"68dd2b1c-ed84-4291-8fb7-f61779fd5b85\") "
Sep 17 14:32:08.848640 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:08.848492    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-istiod-ca-cert\") pod \"68dd2b1c-ed84-4291-8fb7-f61779fd5b85\" (UID: \"68dd2b1c-ed84-4291-8fb7-f61779fd5b85\") "
Sep 17 14:32:08.848640 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:08.848510    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-istio-data\") pod \"68dd2b1c-ed84-4291-8fb7-f61779fd5b85\" (UID: \"68dd2b1c-ed84-4291-8fb7-f61779fd5b85\") "
Sep 17 14:32:08.848640 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:08.848558    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-workload-certs\") pod \"68dd2b1c-ed84-4291-8fb7-f61779fd5b85\" (UID: \"68dd2b1c-ed84-4291-8fb7-f61779fd5b85\") "
Sep 17 14:32:08.848640 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:08.848582    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-istio-token\") pod \"68dd2b1c-ed84-4291-8fb7-f61779fd5b85\" (UID: \"68dd2b1c-ed84-4291-8fb7-f61779fd5b85\") "
Sep 17 14:32:08.848640 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:08.848602    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-credential-socket\") pod \"68dd2b1c-ed84-4291-8fb7-f61779fd5b85\" (UID: \"68dd2b1c-ed84-4291-8fb7-f61779fd5b85\") "
Sep 17 14:32:08.849005 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:08.848677    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-workload-socket\") pod \"68dd2b1c-ed84-4291-8fb7-f61779fd5b85\" (UID: \"68dd2b1c-ed84-4291-8fb7-f61779fd5b85\") "
Sep 17 14:32:08.849005 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:08.848731    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vmp7k\" (UniqueName: \"kubernetes.io/projected/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-kube-api-access-vmp7k\") pod \"68dd2b1c-ed84-4291-8fb7-f61779fd5b85\" (UID: \"68dd2b1c-ed84-4291-8fb7-f61779fd5b85\") "
Sep 17 14:32:08.849005 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:08.848886    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-istio-data" (OuterVolumeSpecName: "istio-data") pod "68dd2b1c-ed84-4291-8fb7-f61779fd5b85" (UID: "68dd2b1c-ed84-4291-8fb7-f61779fd5b85"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:32:08.849005 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:08.848959    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "68dd2b1c-ed84-4291-8fb7-f61779fd5b85" (UID: "68dd2b1c-ed84-4291-8fb7-f61779fd5b85"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:32:08.849005 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:08.848958    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "68dd2b1c-ed84-4291-8fb7-f61779fd5b85" (UID: "68dd2b1c-ed84-4291-8fb7-f61779fd5b85"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:32:08.849256 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:08.849116    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "68dd2b1c-ed84-4291-8fb7-f61779fd5b85" (UID: "68dd2b1c-ed84-4291-8fb7-f61779fd5b85"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:32:08.849256 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:08.849167    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "68dd2b1c-ed84-4291-8fb7-f61779fd5b85" (UID: "68dd2b1c-ed84-4291-8fb7-f61779fd5b85"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:32:08.849256 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:08.849188    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-istiod-ca-cert\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:32:08.849256 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:08.849203    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-istio-data\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:32:08.849256 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:08.849212    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-workload-certs\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:32:08.850888 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:08.850860    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "68dd2b1c-ed84-4291-8fb7-f61779fd5b85" (UID: "68dd2b1c-ed84-4291-8fb7-f61779fd5b85"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:32:08.850888 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:08.850879    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-istio-token" (OuterVolumeSpecName: "istio-token") pod "68dd2b1c-ed84-4291-8fb7-f61779fd5b85" (UID: "68dd2b1c-ed84-4291-8fb7-f61779fd5b85"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:32:08.851058 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:08.850860    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "68dd2b1c-ed84-4291-8fb7-f61779fd5b85" (UID: "68dd2b1c-ed84-4291-8fb7-f61779fd5b85"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 14:32:08.851058 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:08.850969    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-kube-api-access-vmp7k" (OuterVolumeSpecName: "kube-api-access-vmp7k") pod "68dd2b1c-ed84-4291-8fb7-f61779fd5b85" (UID: "68dd2b1c-ed84-4291-8fb7-f61779fd5b85"). InnerVolumeSpecName "kube-api-access-vmp7k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:32:08.950155 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:08.950118    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-istio-token\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:32:08.950155 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:08.950151    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-credential-socket\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:32:08.950365 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:08.950171    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-workload-socket\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:32:08.950365 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:08.950181    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vmp7k\" (UniqueName: \"kubernetes.io/projected/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-kube-api-access-vmp7k\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:32:08.950365 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:08.950189    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-istio-podinfo\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:32:08.950365 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:08.950197    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68dd2b1c-ed84-4291-8fb7-f61779fd5b85-istio-envoy\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:32:09.371592 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:09.371558    2582 generic.go:356] "Generic (PLEG): container finished" podID="68dd2b1c-ed84-4291-8fb7-f61779fd5b85" containerID="e1a29482fea6e73c670a158f9ff4259d158ff57276aa747806fb8ea0d330d790" exitCode=0
Sep 17 14:32:09.371831 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:09.371636    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md"
Sep 17 14:32:09.371831 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:09.371647    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md" event={"ID":"68dd2b1c-ed84-4291-8fb7-f61779fd5b85","Type":"ContainerDied","Data":"e1a29482fea6e73c670a158f9ff4259d158ff57276aa747806fb8ea0d330d790"}
Sep 17 14:32:09.371831 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:09.371704    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md" event={"ID":"68dd2b1c-ed84-4291-8fb7-f61779fd5b85","Type":"ContainerDied","Data":"4af29a761f2bd4c594e1d5f38d34530762379f227551e2c851af1bb00b1578b3"}
Sep 17 14:32:09.371831 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:09.371727    2582 scope.go:117] "RemoveContainer" containerID="e1a29482fea6e73c670a158f9ff4259d158ff57276aa747806fb8ea0d330d790"
Sep 17 14:32:09.382933 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:09.382917    2582 scope.go:117] "RemoveContainer" containerID="ed106bce24d830f2308cc17efa0004d937f823952dc96ce55a4beb200a5caf86"
Sep 17 14:32:09.391581 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:09.391563    2582 scope.go:117] "RemoveContainer" containerID="e1a29482fea6e73c670a158f9ff4259d158ff57276aa747806fb8ea0d330d790"
Sep 17 14:32:09.391886 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:32:09.391866    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1a29482fea6e73c670a158f9ff4259d158ff57276aa747806fb8ea0d330d790\": container with ID starting with e1a29482fea6e73c670a158f9ff4259d158ff57276aa747806fb8ea0d330d790 not found: ID does not exist" containerID="e1a29482fea6e73c670a158f9ff4259d158ff57276aa747806fb8ea0d330d790"
Sep 17 14:32:09.391941 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:09.391896    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1a29482fea6e73c670a158f9ff4259d158ff57276aa747806fb8ea0d330d790"} err="failed to get container status \"e1a29482fea6e73c670a158f9ff4259d158ff57276aa747806fb8ea0d330d790\": rpc error: code = NotFound desc = could not find container \"e1a29482fea6e73c670a158f9ff4259d158ff57276aa747806fb8ea0d330d790\": container with ID starting with e1a29482fea6e73c670a158f9ff4259d158ff57276aa747806fb8ea0d330d790 not found: ID does not exist"
Sep 17 14:32:09.391941 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:09.391915    2582 scope.go:117] "RemoveContainer" containerID="ed106bce24d830f2308cc17efa0004d937f823952dc96ce55a4beb200a5caf86"
Sep 17 14:32:09.392182 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:32:09.392163    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed106bce24d830f2308cc17efa0004d937f823952dc96ce55a4beb200a5caf86\": container with ID starting with ed106bce24d830f2308cc17efa0004d937f823952dc96ce55a4beb200a5caf86 not found: ID does not exist" containerID="ed106bce24d830f2308cc17efa0004d937f823952dc96ce55a4beb200a5caf86"
Sep 17 14:32:09.392228 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:09.392191    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed106bce24d830f2308cc17efa0004d937f823952dc96ce55a4beb200a5caf86"} err="failed to get container status \"ed106bce24d830f2308cc17efa0004d937f823952dc96ce55a4beb200a5caf86\": rpc error: code = NotFound desc = could not find container \"ed106bce24d830f2308cc17efa0004d937f823952dc96ce55a4beb200a5caf86\": container with ID starting with ed106bce24d830f2308cc17efa0004d937f823952dc96ce55a4beb200a5caf86 not found: ID does not exist"
Sep 17 14:32:09.398284 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:09.398258    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md"]
Sep 17 14:32:09.407124 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:09.407085    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-1269b2-openshift-default-7d5fd5748d-ss8md"]
Sep 17 14:32:09.494459 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:09.494422    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68dd2b1c-ed84-4291-8fb7-f61779fd5b85" path="/var/lib/kubelet/pods/68dd2b1c-ed84-4291-8fb7-f61779fd5b85/volumes"
Sep 17 14:32:09.881976 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:09.881940    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:32:12.880138 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:12.880100    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-b83f70-29827590-knbmv"]
Sep 17 14:32:12.882461 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:12.882420    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-b83f70-29827590-knbmv"]
Sep 17 14:32:12.940317 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:12.940282    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:32:13.079805 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:13.079765    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b83f70-75h8b"]
Sep 17 14:32:13.082144 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:13.082115    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b83f70-75h8b"]
Sep 17 14:32:13.087470 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:13.087445    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:32:13.229357 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:13.229321    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:32:13.478089 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:13.478053    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:32:13.496252 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:13.496169    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a376b4d-87db-4a35-b4a7-d0e34ab90b47" path="/var/lib/kubelet/pods/3a376b4d-87db-4a35-b4a7-d0e34ab90b47/volumes"
Sep 17 14:32:13.496686 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:13.496665    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7daa0a3-8adf-4f0f-abdb-85d35779e02e" path="/var/lib/kubelet/pods/a7daa0a3-8adf-4f0f-abdb-85d35779e02e/volumes"
Sep 17 14:32:13.598203 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:13.598170    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:32:14.473870 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:14.473837    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:32:15.247133 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:15.247097    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:32:15.863261 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:15.863225    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:32:19.649479 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:19.649414    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:32:19.796345 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:19.796309    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:32:19.867191 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:19.867155    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr"]
Sep 17 14:32:19.867438 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:19.867409    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr" podUID="932dad53-ff0a-412d-a874-55e0edc95702" containerName="istio-proxy" containerID="cri-o://4f11e69f4e1609715fb3b22835969aadcb007f572a47e85ad06145e6ca0cb248" gracePeriod=30
Sep 17 14:32:19.904573 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:19.904492    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:32:20.794199 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:20.794164    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:32:21.997195 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:21.997164    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:32:22.155845 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:22.155799    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:32:23.176239 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:23.176202    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:32:25.135544 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:25.135477    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr"
Sep 17 14:32:25.219778 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:25.219736    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/932dad53-ff0a-412d-a874-55e0edc95702-istio-data\") pod \"932dad53-ff0a-412d-a874-55e0edc95702\" (UID: \"932dad53-ff0a-412d-a874-55e0edc95702\") "
Sep 17 14:32:25.219979 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:25.219792    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/932dad53-ff0a-412d-a874-55e0edc95702-workload-socket\") pod \"932dad53-ff0a-412d-a874-55e0edc95702\" (UID: \"932dad53-ff0a-412d-a874-55e0edc95702\") "
Sep 17 14:32:25.219979 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:25.219811    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/932dad53-ff0a-412d-a874-55e0edc95702-istio-envoy\") pod \"932dad53-ff0a-412d-a874-55e0edc95702\" (UID: \"932dad53-ff0a-412d-a874-55e0edc95702\") "
Sep 17 14:32:25.219979 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:25.219849    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/932dad53-ff0a-412d-a874-55e0edc95702-istiod-ca-cert\") pod \"932dad53-ff0a-412d-a874-55e0edc95702\" (UID: \"932dad53-ff0a-412d-a874-55e0edc95702\") "
Sep 17 14:32:25.219979 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:25.219888    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/932dad53-ff0a-412d-a874-55e0edc95702-credential-socket\") pod \"932dad53-ff0a-412d-a874-55e0edc95702\" (UID: \"932dad53-ff0a-412d-a874-55e0edc95702\") "
Sep 17 14:32:25.219979 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:25.219920    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/932dad53-ff0a-412d-a874-55e0edc95702-workload-certs\") pod \"932dad53-ff0a-412d-a874-55e0edc95702\" (UID: \"932dad53-ff0a-412d-a874-55e0edc95702\") "
Sep 17 14:32:25.220253 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:25.219987    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9hnq7\" (UniqueName: \"kubernetes.io/projected/932dad53-ff0a-412d-a874-55e0edc95702-kube-api-access-9hnq7\") pod \"932dad53-ff0a-412d-a874-55e0edc95702\" (UID: \"932dad53-ff0a-412d-a874-55e0edc95702\") "
Sep 17 14:32:25.220253 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:25.220029    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/932dad53-ff0a-412d-a874-55e0edc95702-istio-token\") pod \"932dad53-ff0a-412d-a874-55e0edc95702\" (UID: \"932dad53-ff0a-412d-a874-55e0edc95702\") "
Sep 17 14:32:25.220253 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:25.220032    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/932dad53-ff0a-412d-a874-55e0edc95702-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "932dad53-ff0a-412d-a874-55e0edc95702" (UID: "932dad53-ff0a-412d-a874-55e0edc95702"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:32:25.220253 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:25.220046    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/932dad53-ff0a-412d-a874-55e0edc95702-istio-data" (OuterVolumeSpecName: "istio-data") pod "932dad53-ff0a-412d-a874-55e0edc95702" (UID: "932dad53-ff0a-412d-a874-55e0edc95702"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:32:25.220253 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:25.220064    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/932dad53-ff0a-412d-a874-55e0edc95702-istio-podinfo\") pod \"932dad53-ff0a-412d-a874-55e0edc95702\" (UID: \"932dad53-ff0a-412d-a874-55e0edc95702\") "
Sep 17 14:32:25.220253 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:25.220203    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/932dad53-ff0a-412d-a874-55e0edc95702-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "932dad53-ff0a-412d-a874-55e0edc95702" (UID: "932dad53-ff0a-412d-a874-55e0edc95702"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:32:25.220557 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:25.220284    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/932dad53-ff0a-412d-a874-55e0edc95702-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "932dad53-ff0a-412d-a874-55e0edc95702" (UID: "932dad53-ff0a-412d-a874-55e0edc95702"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:32:25.220557 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:25.220331    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/932dad53-ff0a-412d-a874-55e0edc95702-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "932dad53-ff0a-412d-a874-55e0edc95702" (UID: "932dad53-ff0a-412d-a874-55e0edc95702"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:32:25.220557 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:25.220510    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/932dad53-ff0a-412d-a874-55e0edc95702-istiod-ca-cert\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:32:25.220557 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:25.220531    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/932dad53-ff0a-412d-a874-55e0edc95702-credential-socket\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:32:25.220557 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:25.220547    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/932dad53-ff0a-412d-a874-55e0edc95702-workload-certs\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:32:25.220804 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:25.220561    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/932dad53-ff0a-412d-a874-55e0edc95702-istio-data\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:32:25.220804 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:25.220574    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/932dad53-ff0a-412d-a874-55e0edc95702-workload-socket\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:32:25.222240 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:25.222214    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/932dad53-ff0a-412d-a874-55e0edc95702-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "932dad53-ff0a-412d-a874-55e0edc95702" (UID: "932dad53-ff0a-412d-a874-55e0edc95702"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 14:32:25.222240 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:25.222234    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/932dad53-ff0a-412d-a874-55e0edc95702-istio-token" (OuterVolumeSpecName: "istio-token") pod "932dad53-ff0a-412d-a874-55e0edc95702" (UID: "932dad53-ff0a-412d-a874-55e0edc95702"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:32:25.222538 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:25.222516    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/932dad53-ff0a-412d-a874-55e0edc95702-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "932dad53-ff0a-412d-a874-55e0edc95702" (UID: "932dad53-ff0a-412d-a874-55e0edc95702"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:32:25.222538 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:25.222530    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/932dad53-ff0a-412d-a874-55e0edc95702-kube-api-access-9hnq7" (OuterVolumeSpecName: "kube-api-access-9hnq7") pod "932dad53-ff0a-412d-a874-55e0edc95702" (UID: "932dad53-ff0a-412d-a874-55e0edc95702"). InnerVolumeSpecName "kube-api-access-9hnq7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:32:25.321426 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:25.321386    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9hnq7\" (UniqueName: \"kubernetes.io/projected/932dad53-ff0a-412d-a874-55e0edc95702-kube-api-access-9hnq7\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:32:25.321426 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:25.321421    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/932dad53-ff0a-412d-a874-55e0edc95702-istio-token\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:32:25.321426 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:25.321431    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/932dad53-ff0a-412d-a874-55e0edc95702-istio-podinfo\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:32:25.321706 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:25.321439    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/932dad53-ff0a-412d-a874-55e0edc95702-istio-envoy\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:32:25.483896 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:25.483858    2582 generic.go:356] "Generic (PLEG): container finished" podID="932dad53-ff0a-412d-a874-55e0edc95702" containerID="4f11e69f4e1609715fb3b22835969aadcb007f572a47e85ad06145e6ca0cb248" exitCode=0
Sep 17 14:32:25.484068 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:25.483907    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr"
Sep 17 14:32:25.484068 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:25.483922    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr" event={"ID":"932dad53-ff0a-412d-a874-55e0edc95702","Type":"ContainerDied","Data":"4f11e69f4e1609715fb3b22835969aadcb007f572a47e85ad06145e6ca0cb248"}
Sep 17 14:32:25.484068 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:25.483972    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr" event={"ID":"932dad53-ff0a-412d-a874-55e0edc95702","Type":"ContainerDied","Data":"a7223bbbcf1b0a927e3f69ed9130efbbaae18fc6898640f3dc9d147c8c4392bc"}
Sep 17 14:32:25.484068 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:25.483996    2582 scope.go:117] "RemoveContainer" containerID="4f11e69f4e1609715fb3b22835969aadcb007f572a47e85ad06145e6ca0cb248"
Sep 17 14:32:25.495549 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:25.495533    2582 scope.go:117] "RemoveContainer" containerID="d62d9780538b39ced2431aa400c5c9dd51a0c22d35662bbaa79ac53a819a1117"
Sep 17 14:32:25.504855 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:25.504837    2582 scope.go:117] "RemoveContainer" containerID="4f11e69f4e1609715fb3b22835969aadcb007f572a47e85ad06145e6ca0cb248"
Sep 17 14:32:25.505175 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:32:25.505143    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f11e69f4e1609715fb3b22835969aadcb007f572a47e85ad06145e6ca0cb248\": container with ID starting with 4f11e69f4e1609715fb3b22835969aadcb007f572a47e85ad06145e6ca0cb248 not found: ID does not exist" containerID="4f11e69f4e1609715fb3b22835969aadcb007f572a47e85ad06145e6ca0cb248"
Sep 17 14:32:25.505320 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:25.505214    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f11e69f4e1609715fb3b22835969aadcb007f572a47e85ad06145e6ca0cb248"} err="failed to get container status \"4f11e69f4e1609715fb3b22835969aadcb007f572a47e85ad06145e6ca0cb248\": rpc error: code = NotFound desc = could not find container \"4f11e69f4e1609715fb3b22835969aadcb007f572a47e85ad06145e6ca0cb248\": container with ID starting with 4f11e69f4e1609715fb3b22835969aadcb007f572a47e85ad06145e6ca0cb248 not found: ID does not exist"
Sep 17 14:32:25.505320 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:25.505251    2582 scope.go:117] "RemoveContainer" containerID="d62d9780538b39ced2431aa400c5c9dd51a0c22d35662bbaa79ac53a819a1117"
Sep 17 14:32:25.506228 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:32:25.506178    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d62d9780538b39ced2431aa400c5c9dd51a0c22d35662bbaa79ac53a819a1117\": container with ID starting with d62d9780538b39ced2431aa400c5c9dd51a0c22d35662bbaa79ac53a819a1117 not found: ID does not exist" containerID="d62d9780538b39ced2431aa400c5c9dd51a0c22d35662bbaa79ac53a819a1117"
Sep 17 14:32:25.506348 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:25.506233    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d62d9780538b39ced2431aa400c5c9dd51a0c22d35662bbaa79ac53a819a1117"} err="failed to get container status \"d62d9780538b39ced2431aa400c5c9dd51a0c22d35662bbaa79ac53a819a1117\": rpc error: code = NotFound desc = could not find container \"d62d9780538b39ced2431aa400c5c9dd51a0c22d35662bbaa79ac53a819a1117\": container with ID starting with d62d9780538b39ced2431aa400c5c9dd51a0c22d35662bbaa79ac53a819a1117 not found: ID does not exist"
Sep 17 14:32:25.508420 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:25.508397    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr"]
Sep 17 14:32:25.512622 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:25.512590    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-b83f70-openshift-default-76b65b48fb-xclmr"]
Sep 17 14:32:27.494829 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:27.494791    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="932dad53-ff0a-412d-a874-55e0edc95702" path="/var/lib/kubelet/pods/932dad53-ff0a-412d-a874-55e0edc95702/volumes"
Sep 17 14:32:29.140741 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:29.140705    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:32:34.616821 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:34.616780    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:32:57.347567 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:57.347527    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:32:57.432358 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:57.432321    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-fcd9f3-2xsnk"]
Sep 17 14:32:57.432817 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:57.432804    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7daa0a3-8adf-4f0f-abdb-85d35779e02e" containerName="revoke-keys"
Sep 17 14:32:57.432871 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:57.432819    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7daa0a3-8adf-4f0f-abdb-85d35779e02e" containerName="revoke-keys"
Sep 17 14:32:57.432871 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:57.432828    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="932dad53-ff0a-412d-a874-55e0edc95702" containerName="istio-proxy"
Sep 17 14:32:57.432871 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:57.432834    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="932dad53-ff0a-412d-a874-55e0edc95702" containerName="istio-proxy"
Sep 17 14:32:57.432871 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:57.432841    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="932dad53-ff0a-412d-a874-55e0edc95702" containerName="istio-proxy"
Sep 17 14:32:57.432871 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:57.432847    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="932dad53-ff0a-412d-a874-55e0edc95702" containerName="istio-proxy"
Sep 17 14:32:57.432871 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:57.432858    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68dd2b1c-ed84-4291-8fb7-f61779fd5b85" containerName="istio-proxy"
Sep 17 14:32:57.432871 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:57.432863    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="68dd2b1c-ed84-4291-8fb7-f61779fd5b85" containerName="istio-proxy"
Sep 17 14:32:57.432871 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:57.432869    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68dd2b1c-ed84-4291-8fb7-f61779fd5b85" containerName="istio-proxy"
Sep 17 14:32:57.432871 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:57.432875    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="68dd2b1c-ed84-4291-8fb7-f61779fd5b85" containerName="istio-proxy"
Sep 17 14:32:57.433217 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:57.432938    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="68dd2b1c-ed84-4291-8fb7-f61779fd5b85" containerName="istio-proxy"
Sep 17 14:32:57.433217 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:57.432945    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="932dad53-ff0a-412d-a874-55e0edc95702" containerName="istio-proxy"
Sep 17 14:32:57.433217 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:57.432951    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="932dad53-ff0a-412d-a874-55e0edc95702" containerName="istio-proxy"
Sep 17 14:32:57.433217 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:57.432958    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7daa0a3-8adf-4f0f-abdb-85d35779e02e" containerName="revoke-keys"
Sep 17 14:32:57.436277 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:57.436260    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-fcd9f3-2xsnk"
Sep 17 14:32:57.438566 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:57.438546    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ddwrn\""
Sep 17 14:32:57.444226 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:57.444203    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-fcd9f3-2xsnk"]
Sep 17 14:32:57.463357 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:57.463322    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a035404c-dad3-4c44-9d8a-d816be9f0550-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-fcd9f3-2xsnk\" (UID: \"a035404c-dad3-4c44-9d8a-d816be9f0550\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-fcd9f3-2xsnk"
Sep 17 14:32:57.564928 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:57.564893    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a035404c-dad3-4c44-9d8a-d816be9f0550-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-fcd9f3-2xsnk\" (UID: \"a035404c-dad3-4c44-9d8a-d816be9f0550\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-fcd9f3-2xsnk"
Sep 17 14:32:57.565508 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:57.565486    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a035404c-dad3-4c44-9d8a-d816be9f0550-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-fcd9f3-2xsnk\" (UID: \"a035404c-dad3-4c44-9d8a-d816be9f0550\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-fcd9f3-2xsnk"
Sep 17 14:32:57.748419 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:57.748381    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-fcd9f3-2xsnk"
Sep 17 14:32:57.819489 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:57.819445    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:32:58.021292 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:58.021259    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-fcd9f3-2xsnk"]
Sep 17 14:32:58.023644 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:32:58.023586    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda035404c_dad3_4c44_9d8a_d816be9f0550.slice/crio-f9367f8edb77e29efd457af95ac67d61febd18a2b7a3c922410d678b3ec12f93 WatchSource:0}: Error finding container f9367f8edb77e29efd457af95ac67d61febd18a2b7a3c922410d678b3ec12f93: Status 404 returned error can't find the container with id f9367f8edb77e29efd457af95ac67d61febd18a2b7a3c922410d678b3ec12f93
Sep 17 14:32:58.547191 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:58.547160    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:32:58.702220 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:58.702187    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-fcd9f3-2xsnk" event={"ID":"a035404c-dad3-4c44-9d8a-d816be9f0550","Type":"ContainerStarted","Data":"2d661b208c2934ede0a6f9f1ba0872340820efce48b0eb2a71d57a7e3c598b74"}
Sep 17 14:32:58.702393 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:58.702236    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-fcd9f3-2xsnk" event={"ID":"a035404c-dad3-4c44-9d8a-d816be9f0550","Type":"ContainerStarted","Data":"f9367f8edb77e29efd457af95ac67d61febd18a2b7a3c922410d678b3ec12f93"}
Sep 17 14:32:58.718844 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:58.718795    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-fcd9f3-2xsnk" podStartSLOduration=1.718776176 podStartE2EDuration="1.718776176s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:32:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:32:58.717779323 +0000 UTC m=+1679.719322797" watchObservedRunningTime="2026-09-17 14:32:58.718776176 +0000 UTC m=+1679.720319654"
Sep 17 14:32:59.329105 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:59.329067    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:32:59.709374 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:59.709338    2582 generic.go:356] "Generic (PLEG): container finished" podID="a035404c-dad3-4c44-9d8a-d816be9f0550" containerID="2d661b208c2934ede0a6f9f1ba0872340820efce48b0eb2a71d57a7e3c598b74" exitCode=0
Sep 17 14:32:59.709755 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:59.709418    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-fcd9f3-2xsnk" event={"ID":"a035404c-dad3-4c44-9d8a-d816be9f0550","Type":"ContainerDied","Data":"2d661b208c2934ede0a6f9f1ba0872340820efce48b0eb2a71d57a7e3c598b74"}
Sep 17 14:32:59.822012 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:32:59.821981    2582 scope.go:117] "RemoveContainer" containerID="4cbc6fd62788bf54bd8551df69525166e8b98aea0d7aaa558fee592a3752c3af"
Sep 17 14:33:00.040031 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:00.039938    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:33:00.830951 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:00.830919    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:33:00.869559 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:00.869529    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-fcd9f3-2xsnk"
Sep 17 14:33:00.895576 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:00.895543    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a035404c-dad3-4c44-9d8a-d816be9f0550-maas-api-service-ca\") pod \"a035404c-dad3-4c44-9d8a-d816be9f0550\" (UID: \"a035404c-dad3-4c44-9d8a-d816be9f0550\") "
Sep 17 14:33:00.895889 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:00.895869    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a035404c-dad3-4c44-9d8a-d816be9f0550-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a035404c-dad3-4c44-9d8a-d816be9f0550" (UID: "a035404c-dad3-4c44-9d8a-d816be9f0550"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:33:00.906286 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:00.896473    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a035404c-dad3-4c44-9d8a-d816be9f0550-maas-api-service-ca\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:33:01.723404 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:01.723376    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-fcd9f3-2xsnk"
Sep 17 14:33:01.723575 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:01.723413    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-fcd9f3-2xsnk" event={"ID":"a035404c-dad3-4c44-9d8a-d816be9f0550","Type":"ContainerDied","Data":"f9367f8edb77e29efd457af95ac67d61febd18a2b7a3c922410d678b3ec12f93"}
Sep 17 14:33:01.723575 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:01.723464    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f9367f8edb77e29efd457af95ac67d61febd18a2b7a3c922410d678b3ec12f93"
Sep 17 14:33:02.415439 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:02.415403    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-fcd9f3/distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn"]
Sep 17 14:33:02.602441 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:02.602405    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:33:07.185818 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:07.185776    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:33:07.978157 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:07.978121    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:33:13.348175 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:13.348139    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-fcd9f3-29827590-dfxb7"]
Sep 17 14:33:13.352801 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:13.352772    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-fcd9f3-29827590-dfxb7"]
Sep 17 14:33:13.364491 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:13.364465    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:33:13.475356 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:13.475314    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:33:13.493867 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:13.493838    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a92a7aa0-ad51-4c3c-a0b0-a25972496d3a" path="/var/lib/kubelet/pods/a92a7aa0-ad51-4c3c-a0b0-a25972496d3a/volumes"
Sep 17 14:33:13.530946 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:13.530907    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-fcd9f3-2xsnk"]
Sep 17 14:33:13.534008 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:13.533987    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-fcd9f3-2xsnk"]
Sep 17 14:33:13.580189 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:13.580161    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:33:13.908417 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:13.908387    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:33:14.028809 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:14.028776    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:33:14.672262 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:14.672219    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:33:15.494233 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:15.494187    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a035404c-dad3-4c44-9d8a-d816be9f0550" path="/var/lib/kubelet/pods/a035404c-dad3-4c44-9d8a-d816be9f0550/volumes"
Sep 17 14:33:15.544075 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:15.544039    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:33:17.436191 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:17.436152    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-fcd9f3/distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn" podUID="775a4af7-36fe-429a-9fde-d85bc8ce5cfb" containerName="main" containerID="cri-o://c50f45b7f62bea17926765fbbeb75eb036fc32d8325a2f72cf8242ab150b1c77" gracePeriod=45
Sep 17 14:33:17.712219 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:17.712194    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-fcd9f3/distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn"
Sep 17 14:33:17.778005 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:17.777959    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/775a4af7-36fe-429a-9fde-d85bc8ce5cfb-tmp-dir\") pod \"775a4af7-36fe-429a-9fde-d85bc8ce5cfb\" (UID: \"775a4af7-36fe-429a-9fde-d85bc8ce5cfb\") "
Sep 17 14:33:17.778190 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:17.778053    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/775a4af7-36fe-429a-9fde-d85bc8ce5cfb-home\") pod \"775a4af7-36fe-429a-9fde-d85bc8ce5cfb\" (UID: \"775a4af7-36fe-429a-9fde-d85bc8ce5cfb\") "
Sep 17 14:33:17.778190 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:17.778109    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/775a4af7-36fe-429a-9fde-d85bc8ce5cfb-model-cache\") pod \"775a4af7-36fe-429a-9fde-d85bc8ce5cfb\" (UID: \"775a4af7-36fe-429a-9fde-d85bc8ce5cfb\") "
Sep 17 14:33:17.778190 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:17.778144    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/775a4af7-36fe-429a-9fde-d85bc8ce5cfb-dshm\") pod \"775a4af7-36fe-429a-9fde-d85bc8ce5cfb\" (UID: \"775a4af7-36fe-429a-9fde-d85bc8ce5cfb\") "
Sep 17 14:33:17.778190 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:17.778171    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/775a4af7-36fe-429a-9fde-d85bc8ce5cfb-tls-certs\") pod \"775a4af7-36fe-429a-9fde-d85bc8ce5cfb\" (UID: \"775a4af7-36fe-429a-9fde-d85bc8ce5cfb\") "
Sep 17 14:33:17.778432 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:17.778262    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-km5jb\" (UniqueName: \"kubernetes.io/projected/775a4af7-36fe-429a-9fde-d85bc8ce5cfb-kube-api-access-km5jb\") pod \"775a4af7-36fe-429a-9fde-d85bc8ce5cfb\" (UID: \"775a4af7-36fe-429a-9fde-d85bc8ce5cfb\") "
Sep 17 14:33:17.779189 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:17.779159    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/775a4af7-36fe-429a-9fde-d85bc8ce5cfb-model-cache" (OuterVolumeSpecName: "model-cache") pod "775a4af7-36fe-429a-9fde-d85bc8ce5cfb" (UID: "775a4af7-36fe-429a-9fde-d85bc8ce5cfb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:33:17.779525 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:17.779355    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/775a4af7-36fe-429a-9fde-d85bc8ce5cfb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "775a4af7-36fe-429a-9fde-d85bc8ce5cfb" (UID: "775a4af7-36fe-429a-9fde-d85bc8ce5cfb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:33:17.779969 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:17.779696    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/775a4af7-36fe-429a-9fde-d85bc8ce5cfb-home" (OuterVolumeSpecName: "home") pod "775a4af7-36fe-429a-9fde-d85bc8ce5cfb" (UID: "775a4af7-36fe-429a-9fde-d85bc8ce5cfb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:33:17.781929 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:17.781881    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/775a4af7-36fe-429a-9fde-d85bc8ce5cfb-kube-api-access-km5jb" (OuterVolumeSpecName: "kube-api-access-km5jb") pod "775a4af7-36fe-429a-9fde-d85bc8ce5cfb" (UID: "775a4af7-36fe-429a-9fde-d85bc8ce5cfb"). InnerVolumeSpecName "kube-api-access-km5jb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:33:17.784180 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:17.784156    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/775a4af7-36fe-429a-9fde-d85bc8ce5cfb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "775a4af7-36fe-429a-9fde-d85bc8ce5cfb" (UID: "775a4af7-36fe-429a-9fde-d85bc8ce5cfb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:33:17.784300 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:17.784232    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/775a4af7-36fe-429a-9fde-d85bc8ce5cfb-dshm" (OuterVolumeSpecName: "dshm") pod "775a4af7-36fe-429a-9fde-d85bc8ce5cfb" (UID: "775a4af7-36fe-429a-9fde-d85bc8ce5cfb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:33:17.827375 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:17.827335    2582 generic.go:356] "Generic (PLEG): container finished" podID="775a4af7-36fe-429a-9fde-d85bc8ce5cfb" containerID="c50f45b7f62bea17926765fbbeb75eb036fc32d8325a2f72cf8242ab150b1c77" exitCode=0
Sep 17 14:33:17.827555 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:17.827479    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-fcd9f3/distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn" event={"ID":"775a4af7-36fe-429a-9fde-d85bc8ce5cfb","Type":"ContainerDied","Data":"c50f45b7f62bea17926765fbbeb75eb036fc32d8325a2f72cf8242ab150b1c77"}
Sep 17 14:33:17.827555 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:17.827513    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-fcd9f3/distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn" event={"ID":"775a4af7-36fe-429a-9fde-d85bc8ce5cfb","Type":"ContainerDied","Data":"a78762c0009b73a18e8b2ba897e49ddc1feb6ef160941346551699c8f2475a45"}
Sep 17 14:33:17.827555 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:17.827533    2582 scope.go:117] "RemoveContainer" containerID="c50f45b7f62bea17926765fbbeb75eb036fc32d8325a2f72cf8242ab150b1c77"
Sep 17 14:33:17.827755 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:17.827564    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-fcd9f3/distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn"
Sep 17 14:33:17.840414 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:17.840392    2582 scope.go:117] "RemoveContainer" containerID="c50f45b7f62bea17926765fbbeb75eb036fc32d8325a2f72cf8242ab150b1c77"
Sep 17 14:33:17.840743 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:33:17.840716    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c50f45b7f62bea17926765fbbeb75eb036fc32d8325a2f72cf8242ab150b1c77\": container with ID starting with c50f45b7f62bea17926765fbbeb75eb036fc32d8325a2f72cf8242ab150b1c77 not found: ID does not exist" containerID="c50f45b7f62bea17926765fbbeb75eb036fc32d8325a2f72cf8242ab150b1c77"
Sep 17 14:33:17.840835 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:17.840758    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c50f45b7f62bea17926765fbbeb75eb036fc32d8325a2f72cf8242ab150b1c77"} err="failed to get container status \"c50f45b7f62bea17926765fbbeb75eb036fc32d8325a2f72cf8242ab150b1c77\": rpc error: code = NotFound desc = could not find container \"c50f45b7f62bea17926765fbbeb75eb036fc32d8325a2f72cf8242ab150b1c77\": container with ID starting with c50f45b7f62bea17926765fbbeb75eb036fc32d8325a2f72cf8242ab150b1c77 not found: ID does not exist"
Sep 17 14:33:17.857405 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:17.857377    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-fcd9f3/distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn"]
Sep 17 14:33:17.861597 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:17.861575    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-fcd9f3/distinct-w4-fcd9f3-kserve-8499dcbb56-gthkn"]
Sep 17 14:33:17.879531 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:17.879509    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/775a4af7-36fe-429a-9fde-d85bc8ce5cfb-model-cache\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:33:17.879531 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:17.879531    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/775a4af7-36fe-429a-9fde-d85bc8ce5cfb-dshm\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:33:17.879749 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:17.879540    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/775a4af7-36fe-429a-9fde-d85bc8ce5cfb-tls-certs\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:33:17.879749 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:17.879548    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-km5jb\" (UniqueName: \"kubernetes.io/projected/775a4af7-36fe-429a-9fde-d85bc8ce5cfb-kube-api-access-km5jb\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:33:17.879749 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:17.879557    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/775a4af7-36fe-429a-9fde-d85bc8ce5cfb-tmp-dir\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:33:17.879749 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:17.879565    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/775a4af7-36fe-429a-9fde-d85bc8ce5cfb-home\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:33:19.494314 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:19.494277    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="775a4af7-36fe-429a-9fde-d85bc8ce5cfb" path="/var/lib/kubelet/pods/775a4af7-36fe-429a-9fde-d85bc8ce5cfb/volumes"
Sep 17 14:33:19.755265 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:19.755178    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:33:23.661389 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:23.661294    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:33:23.812811 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:23.812771    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:33:25.131623 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:25.131558    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:33:38.272352 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:38.272318    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-cc3801/auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9"]
Sep 17 14:33:39.279124 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:39.279088    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-4c29a2/auth-test-model-4c29a2-kserve-8c7d98b47-6tn75"]
Sep 17 14:33:53.291675 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:53.291631    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-cc3801/auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9" podUID="4d2f89a0-b1b2-47d9-976a-2435cf66750a" containerName="main" containerID="cri-o://54867acb7e8999cd1c702c44d13ca482a98434bb4acd7df85ef7387d30440a4a" gracePeriod=45
Sep 17 14:33:53.550737 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:53.550672    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-cc3801/auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9"
Sep 17 14:33:53.642860 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:53.642815    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p778h\" (UniqueName: \"kubernetes.io/projected/4d2f89a0-b1b2-47d9-976a-2435cf66750a-kube-api-access-p778h\") pod \"4d2f89a0-b1b2-47d9-976a-2435cf66750a\" (UID: \"4d2f89a0-b1b2-47d9-976a-2435cf66750a\") "
Sep 17 14:33:53.643053 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:53.642886    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d2f89a0-b1b2-47d9-976a-2435cf66750a-tls-certs\") pod \"4d2f89a0-b1b2-47d9-976a-2435cf66750a\" (UID: \"4d2f89a0-b1b2-47d9-976a-2435cf66750a\") "
Sep 17 14:33:53.643053 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:53.642962    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d2f89a0-b1b2-47d9-976a-2435cf66750a-tmp-dir\") pod \"4d2f89a0-b1b2-47d9-976a-2435cf66750a\" (UID: \"4d2f89a0-b1b2-47d9-976a-2435cf66750a\") "
Sep 17 14:33:53.643053 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:53.643005    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d2f89a0-b1b2-47d9-976a-2435cf66750a-dshm\") pod \"4d2f89a0-b1b2-47d9-976a-2435cf66750a\" (UID: \"4d2f89a0-b1b2-47d9-976a-2435cf66750a\") "
Sep 17 14:33:53.643218 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:53.643060    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d2f89a0-b1b2-47d9-976a-2435cf66750a-model-cache\") pod \"4d2f89a0-b1b2-47d9-976a-2435cf66750a\" (UID: \"4d2f89a0-b1b2-47d9-976a-2435cf66750a\") "
Sep 17 14:33:53.643218 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:53.643074    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d2f89a0-b1b2-47d9-976a-2435cf66750a-home\") pod \"4d2f89a0-b1b2-47d9-976a-2435cf66750a\" (UID: \"4d2f89a0-b1b2-47d9-976a-2435cf66750a\") "
Sep 17 14:33:53.643336 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:53.643225    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d2f89a0-b1b2-47d9-976a-2435cf66750a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4d2f89a0-b1b2-47d9-976a-2435cf66750a" (UID: "4d2f89a0-b1b2-47d9-976a-2435cf66750a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:33:53.643389 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:53.643333    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d2f89a0-b1b2-47d9-976a-2435cf66750a-model-cache" (OuterVolumeSpecName: "model-cache") pod "4d2f89a0-b1b2-47d9-976a-2435cf66750a" (UID: "4d2f89a0-b1b2-47d9-976a-2435cf66750a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:33:53.643560 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:53.643536    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d2f89a0-b1b2-47d9-976a-2435cf66750a-home" (OuterVolumeSpecName: "home") pod "4d2f89a0-b1b2-47d9-976a-2435cf66750a" (UID: "4d2f89a0-b1b2-47d9-976a-2435cf66750a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:33:53.643560 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:53.643553    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d2f89a0-b1b2-47d9-976a-2435cf66750a-tmp-dir\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:33:53.643705 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:53.643573    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d2f89a0-b1b2-47d9-976a-2435cf66750a-model-cache\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:33:53.645215 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:53.645190    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d2f89a0-b1b2-47d9-976a-2435cf66750a-dshm" (OuterVolumeSpecName: "dshm") pod "4d2f89a0-b1b2-47d9-976a-2435cf66750a" (UID: "4d2f89a0-b1b2-47d9-976a-2435cf66750a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:33:53.645314 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:53.645264    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d2f89a0-b1b2-47d9-976a-2435cf66750a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4d2f89a0-b1b2-47d9-976a-2435cf66750a" (UID: "4d2f89a0-b1b2-47d9-976a-2435cf66750a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:33:53.645352 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:53.645327    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d2f89a0-b1b2-47d9-976a-2435cf66750a-kube-api-access-p778h" (OuterVolumeSpecName: "kube-api-access-p778h") pod "4d2f89a0-b1b2-47d9-976a-2435cf66750a" (UID: "4d2f89a0-b1b2-47d9-976a-2435cf66750a"). InnerVolumeSpecName "kube-api-access-p778h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:33:53.744251 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:53.744216    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d2f89a0-b1b2-47d9-976a-2435cf66750a-dshm\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:33:53.744251 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:53.744247    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d2f89a0-b1b2-47d9-976a-2435cf66750a-home\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:33:53.744251 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:53.744257    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p778h\" (UniqueName: \"kubernetes.io/projected/4d2f89a0-b1b2-47d9-976a-2435cf66750a-kube-api-access-p778h\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:33:53.744493 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:53.744266    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d2f89a0-b1b2-47d9-976a-2435cf66750a-tls-certs\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:33:54.055700 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:54.055665    2582 generic.go:356] "Generic (PLEG): container finished" podID="4d2f89a0-b1b2-47d9-976a-2435cf66750a" containerID="54867acb7e8999cd1c702c44d13ca482a98434bb4acd7df85ef7387d30440a4a" exitCode=0
Sep 17 14:33:54.055876 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:54.055707    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-cc3801/auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9"
Sep 17 14:33:54.055876 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:54.055747    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-cc3801/auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9" event={"ID":"4d2f89a0-b1b2-47d9-976a-2435cf66750a","Type":"ContainerDied","Data":"54867acb7e8999cd1c702c44d13ca482a98434bb4acd7df85ef7387d30440a4a"}
Sep 17 14:33:54.055876 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:54.055792    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-cc3801/auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9" event={"ID":"4d2f89a0-b1b2-47d9-976a-2435cf66750a","Type":"ContainerDied","Data":"005189e1074a97990be3c5c190d114590ba0ae8250583a3143b3b4325b798c5c"}
Sep 17 14:33:54.055876 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:54.055811    2582 scope.go:117] "RemoveContainer" containerID="54867acb7e8999cd1c702c44d13ca482a98434bb4acd7df85ef7387d30440a4a"
Sep 17 14:33:54.065565 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:54.065549    2582 scope.go:117] "RemoveContainer" containerID="54867acb7e8999cd1c702c44d13ca482a98434bb4acd7df85ef7387d30440a4a"
Sep 17 14:33:54.065837 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:33:54.065821    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54867acb7e8999cd1c702c44d13ca482a98434bb4acd7df85ef7387d30440a4a\": container with ID starting with 54867acb7e8999cd1c702c44d13ca482a98434bb4acd7df85ef7387d30440a4a not found: ID does not exist" containerID="54867acb7e8999cd1c702c44d13ca482a98434bb4acd7df85ef7387d30440a4a"
Sep 17 14:33:54.065885 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:54.065845    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54867acb7e8999cd1c702c44d13ca482a98434bb4acd7df85ef7387d30440a4a"} err="failed to get container status \"54867acb7e8999cd1c702c44d13ca482a98434bb4acd7df85ef7387d30440a4a\": rpc error: code = NotFound desc = could not find container \"54867acb7e8999cd1c702c44d13ca482a98434bb4acd7df85ef7387d30440a4a\": container with ID starting with 54867acb7e8999cd1c702c44d13ca482a98434bb4acd7df85ef7387d30440a4a not found: ID does not exist"
Sep 17 14:33:54.078451 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:54.078427    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-cc3801/auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9"]
Sep 17 14:33:54.082504 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:54.082481    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-cc3801/auth-test-model-cc3801-kserve-65dfdfccc9-fnmr9"]
Sep 17 14:33:54.298807 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:54.298770    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-4c29a2/auth-test-model-4c29a2-kserve-8c7d98b47-6tn75" podUID="3731cd4a-3fe7-4a2c-8d28-bda2472f1042" containerName="main" containerID="cri-o://5310629e2035b1102f6aef8fbdfbfac36d837eb9a00be5ba3d8e50a34f236d49" gracePeriod=45
Sep 17 14:33:54.574666 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:54.574595    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-4c29a2/auth-test-model-4c29a2-kserve-8c7d98b47-6tn75"
Sep 17 14:33:54.652544 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:54.652511    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7pq7c\" (UniqueName: \"kubernetes.io/projected/3731cd4a-3fe7-4a2c-8d28-bda2472f1042-kube-api-access-7pq7c\") pod \"3731cd4a-3fe7-4a2c-8d28-bda2472f1042\" (UID: \"3731cd4a-3fe7-4a2c-8d28-bda2472f1042\") "
Sep 17 14:33:54.652752 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:54.652584    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3731cd4a-3fe7-4a2c-8d28-bda2472f1042-dshm\") pod \"3731cd4a-3fe7-4a2c-8d28-bda2472f1042\" (UID: \"3731cd4a-3fe7-4a2c-8d28-bda2472f1042\") "
Sep 17 14:33:54.652752 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:54.652602    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3731cd4a-3fe7-4a2c-8d28-bda2472f1042-home\") pod \"3731cd4a-3fe7-4a2c-8d28-bda2472f1042\" (UID: \"3731cd4a-3fe7-4a2c-8d28-bda2472f1042\") "
Sep 17 14:33:54.652752 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:54.652664    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3731cd4a-3fe7-4a2c-8d28-bda2472f1042-tls-certs\") pod \"3731cd4a-3fe7-4a2c-8d28-bda2472f1042\" (UID: \"3731cd4a-3fe7-4a2c-8d28-bda2472f1042\") "
Sep 17 14:33:54.652752 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:54.652692    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3731cd4a-3fe7-4a2c-8d28-bda2472f1042-model-cache\") pod \"3731cd4a-3fe7-4a2c-8d28-bda2472f1042\" (UID: \"3731cd4a-3fe7-4a2c-8d28-bda2472f1042\") "
Sep 17 14:33:54.652752 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:54.652744    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3731cd4a-3fe7-4a2c-8d28-bda2472f1042-tmp-dir\") pod \"3731cd4a-3fe7-4a2c-8d28-bda2472f1042\" (UID: \"3731cd4a-3fe7-4a2c-8d28-bda2472f1042\") "
Sep 17 14:33:54.653037 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:54.652899    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3731cd4a-3fe7-4a2c-8d28-bda2472f1042-home" (OuterVolumeSpecName: "home") pod "3731cd4a-3fe7-4a2c-8d28-bda2472f1042" (UID: "3731cd4a-3fe7-4a2c-8d28-bda2472f1042"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:33:54.653037 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:54.652937    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3731cd4a-3fe7-4a2c-8d28-bda2472f1042-model-cache" (OuterVolumeSpecName: "model-cache") pod "3731cd4a-3fe7-4a2c-8d28-bda2472f1042" (UID: "3731cd4a-3fe7-4a2c-8d28-bda2472f1042"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:33:54.653150 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:54.653038    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3731cd4a-3fe7-4a2c-8d28-bda2472f1042-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3731cd4a-3fe7-4a2c-8d28-bda2472f1042" (UID: "3731cd4a-3fe7-4a2c-8d28-bda2472f1042"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:33:54.653198 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:54.653163    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3731cd4a-3fe7-4a2c-8d28-bda2472f1042-home\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:33:54.653198 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:54.653183    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3731cd4a-3fe7-4a2c-8d28-bda2472f1042-model-cache\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:33:54.653198 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:54.653195    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3731cd4a-3fe7-4a2c-8d28-bda2472f1042-tmp-dir\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:33:54.654903 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:54.654879    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3731cd4a-3fe7-4a2c-8d28-bda2472f1042-kube-api-access-7pq7c" (OuterVolumeSpecName: "kube-api-access-7pq7c") pod "3731cd4a-3fe7-4a2c-8d28-bda2472f1042" (UID: "3731cd4a-3fe7-4a2c-8d28-bda2472f1042"). InnerVolumeSpecName "kube-api-access-7pq7c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:33:54.655135 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:54.655117    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3731cd4a-3fe7-4a2c-8d28-bda2472f1042-dshm" (OuterVolumeSpecName: "dshm") pod "3731cd4a-3fe7-4a2c-8d28-bda2472f1042" (UID: "3731cd4a-3fe7-4a2c-8d28-bda2472f1042"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:33:54.655193 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:54.655130    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3731cd4a-3fe7-4a2c-8d28-bda2472f1042-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3731cd4a-3fe7-4a2c-8d28-bda2472f1042" (UID: "3731cd4a-3fe7-4a2c-8d28-bda2472f1042"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:33:54.753828 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:54.753793    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7pq7c\" (UniqueName: \"kubernetes.io/projected/3731cd4a-3fe7-4a2c-8d28-bda2472f1042-kube-api-access-7pq7c\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:33:54.753828 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:54.753824    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3731cd4a-3fe7-4a2c-8d28-bda2472f1042-dshm\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:33:54.753828 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:54.753835    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3731cd4a-3fe7-4a2c-8d28-bda2472f1042-tls-certs\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:33:55.064695 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:55.064634    2582 generic.go:356] "Generic (PLEG): container finished" podID="3731cd4a-3fe7-4a2c-8d28-bda2472f1042" containerID="5310629e2035b1102f6aef8fbdfbfac36d837eb9a00be5ba3d8e50a34f236d49" exitCode=0
Sep 17 14:33:55.064695 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:55.064685    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-4c29a2/auth-test-model-4c29a2-kserve-8c7d98b47-6tn75"
Sep 17 14:33:55.064922 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:55.064724    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-4c29a2/auth-test-model-4c29a2-kserve-8c7d98b47-6tn75" event={"ID":"3731cd4a-3fe7-4a2c-8d28-bda2472f1042","Type":"ContainerDied","Data":"5310629e2035b1102f6aef8fbdfbfac36d837eb9a00be5ba3d8e50a34f236d49"}
Sep 17 14:33:55.064922 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:55.064771    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-4c29a2/auth-test-model-4c29a2-kserve-8c7d98b47-6tn75" event={"ID":"3731cd4a-3fe7-4a2c-8d28-bda2472f1042","Type":"ContainerDied","Data":"60b169de097d54703f3e4fb03c5c92d246bc2df14e592cd97776dd60d2223c43"}
Sep 17 14:33:55.064922 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:55.064789    2582 scope.go:117] "RemoveContainer" containerID="5310629e2035b1102f6aef8fbdfbfac36d837eb9a00be5ba3d8e50a34f236d49"
Sep 17 14:33:55.075392 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:55.075372    2582 scope.go:117] "RemoveContainer" containerID="5310629e2035b1102f6aef8fbdfbfac36d837eb9a00be5ba3d8e50a34f236d49"
Sep 17 14:33:55.075706 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:33:55.075682    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5310629e2035b1102f6aef8fbdfbfac36d837eb9a00be5ba3d8e50a34f236d49\": container with ID starting with 5310629e2035b1102f6aef8fbdfbfac36d837eb9a00be5ba3d8e50a34f236d49 not found: ID does not exist" containerID="5310629e2035b1102f6aef8fbdfbfac36d837eb9a00be5ba3d8e50a34f236d49"
Sep 17 14:33:55.075754 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:55.075719    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5310629e2035b1102f6aef8fbdfbfac36d837eb9a00be5ba3d8e50a34f236d49"} err="failed to get container status \"5310629e2035b1102f6aef8fbdfbfac36d837eb9a00be5ba3d8e50a34f236d49\": rpc error: code = NotFound desc = could not find container \"5310629e2035b1102f6aef8fbdfbfac36d837eb9a00be5ba3d8e50a34f236d49\": container with ID starting with 5310629e2035b1102f6aef8fbdfbfac36d837eb9a00be5ba3d8e50a34f236d49 not found: ID does not exist"
Sep 17 14:33:55.088129 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:55.088106    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-4c29a2/auth-test-model-4c29a2-kserve-8c7d98b47-6tn75"]
Sep 17 14:33:55.092376 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:55.092354    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-4c29a2/auth-test-model-4c29a2-kserve-8c7d98b47-6tn75"]
Sep 17 14:33:55.495639 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:55.495590    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3731cd4a-3fe7-4a2c-8d28-bda2472f1042" path="/var/lib/kubelet/pods/3731cd4a-3fe7-4a2c-8d28-bda2472f1042/volumes"
Sep 17 14:33:55.496077 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:55.496051    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d2f89a0-b1b2-47d9-976a-2435cf66750a" path="/var/lib/kubelet/pods/4d2f89a0-b1b2-47d9-976a-2435cf66750a/volumes"
Sep 17 14:33:59.859691 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:33:59.859659    2582 scope.go:117] "RemoveContainer" containerID="a09de9490b39a99a7931a003210011b3f99e186882d8249c8c4e4128118c2295"
Sep 17 14:34:59.608491 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:34:59.608458    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-rr97k_45ffa13d-635a-46e8-8894-30902b9cf719/console-operator/2.log"
Sep 17 14:34:59.612277 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:34:59.612254    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-rr97k_45ffa13d-635a-46e8-8894-30902b9cf719/console-operator/2.log"
Sep 17 14:35:30.472823 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:35:30.472788    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"]
Sep 17 14:35:30.473339 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:35:30.473086    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6" podUID="368ca806-07df-48be-8ee6-3e8773114611" containerName="istio-proxy" containerID="cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09" gracePeriod=30
Sep 17 14:35:30.492021 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:35:30.491985    2582 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"368ca806-07df-48be-8ee6-3e8773114611\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"openshift-ingress\"/\"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/pod?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:35:35.675335 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:35:35.675297    2582 generic.go:356] "Generic (PLEG): container finished" podID="368ca806-07df-48be-8ee6-3e8773114611" containerID="b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09" exitCode=0
Sep 17 14:35:35.675797 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:35:35.675365    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6" event={"ID":"368ca806-07df-48be-8ee6-3e8773114611","Type":"ContainerDied","Data":"b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09"}
Sep 17 14:35:35.675797 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:35:35.675428    2582 scope.go:117] "RemoveContainer" containerID="7000bfc51c007aaa54be164e8b608517d22220b1155dd9ec6af3c4fe1501b0c9"
Sep 17 14:35:35.739743 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:35:35.739675    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:35:35.758247 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:35:35.758167    2582 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"368ca806-07df-48be-8ee6-3e8773114611\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"},\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"image\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"imageID\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"istio-proxy\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"limits\\\":{\\\"cpu\\\":\\\"2\\\",\\\"memory\\\":\\\"1Gi\\\"},\\\"requests\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"}},\\\"restartCount\\\":1,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-09-17T14:19:17Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":1000329999,\\\"supplementalGroups\\\":[1000329999,1000320000],\\\"uid\\\":1000329999}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-uds\\\",\\\"name\\\":\\\"workload-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/credential-uds\\\",\\\"name\\\":\\\"credential-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-credentials\\\",\\\"name\\\":\\\"workload-certs\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/istio\\\",\\\"name\\\":\\\"istiod-ca-cert\\\"},{\\\"mountPath\\\":\\\"/var/lib/istio/data\\\",\\\"name\\\":\\\"istio-data\\\"},{\\\"mountPath\\\":\\\"/etc/istio/proxy\\\",\\\"name\\\":\\\"istio-envoy\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/tokens\\\",\\\"name\\\":\\\"istio-token\\\"},{\\\"mountPath\\\":\\\"/etc/istio/pod\\\",\\\"name\\\":\\\"istio-podinfo\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-7jrjb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"observedGeneration\\\":2,\\\"phase\\\":\\\"Succeeded\\\"}}\" for pod \"openshift-ingress\"/\"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/pod?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:35:35.841042 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:35:35.841011    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/368ca806-07df-48be-8ee6-3e8773114611-istio-envoy\") pod \"368ca806-07df-48be-8ee6-3e8773114611\" (UID: \"368ca806-07df-48be-8ee6-3e8773114611\") "
Sep 17 14:35:35.841221 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:35:35.841055    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/368ca806-07df-48be-8ee6-3e8773114611-workload-socket\") pod \"368ca806-07df-48be-8ee6-3e8773114611\" (UID: \"368ca806-07df-48be-8ee6-3e8773114611\") "
Sep 17 14:35:35.841221 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:35:35.841082    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/368ca806-07df-48be-8ee6-3e8773114611-istio-podinfo\") pod \"368ca806-07df-48be-8ee6-3e8773114611\" (UID: \"368ca806-07df-48be-8ee6-3e8773114611\") "
Sep 17 14:35:35.841221 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:35:35.841111    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/368ca806-07df-48be-8ee6-3e8773114611-istiod-ca-cert\") pod \"368ca806-07df-48be-8ee6-3e8773114611\" (UID: \"368ca806-07df-48be-8ee6-3e8773114611\") "
Sep 17 14:35:35.841221 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:35:35.841149    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7jrjb\" (UniqueName: \"kubernetes.io/projected/368ca806-07df-48be-8ee6-3e8773114611-kube-api-access-7jrjb\") pod \"368ca806-07df-48be-8ee6-3e8773114611\" (UID: \"368ca806-07df-48be-8ee6-3e8773114611\") "
Sep 17 14:35:35.841221 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:35:35.841187    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/368ca806-07df-48be-8ee6-3e8773114611-credential-socket\") pod \"368ca806-07df-48be-8ee6-3e8773114611\" (UID: \"368ca806-07df-48be-8ee6-3e8773114611\") "
Sep 17 14:35:35.841221 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:35:35.841220    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/368ca806-07df-48be-8ee6-3e8773114611-istio-token\") pod \"368ca806-07df-48be-8ee6-3e8773114611\" (UID: \"368ca806-07df-48be-8ee6-3e8773114611\") "
Sep 17 14:35:35.841547 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:35:35.841263    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/368ca806-07df-48be-8ee6-3e8773114611-workload-certs\") pod \"368ca806-07df-48be-8ee6-3e8773114611\" (UID: \"368ca806-07df-48be-8ee6-3e8773114611\") "
Sep 17 14:35:35.841547 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:35:35.841302    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/368ca806-07df-48be-8ee6-3e8773114611-istio-data\") pod \"368ca806-07df-48be-8ee6-3e8773114611\" (UID: \"368ca806-07df-48be-8ee6-3e8773114611\") "
Sep 17 14:35:35.841547 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:35:35.841364    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/368ca806-07df-48be-8ee6-3e8773114611-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "368ca806-07df-48be-8ee6-3e8773114611" (UID: "368ca806-07df-48be-8ee6-3e8773114611"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:35:35.841732 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:35:35.841562    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/368ca806-07df-48be-8ee6-3e8773114611-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "368ca806-07df-48be-8ee6-3e8773114611" (UID: "368ca806-07df-48be-8ee6-3e8773114611"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:35:35.841732 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:35:35.841651    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/368ca806-07df-48be-8ee6-3e8773114611-workload-socket\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:35:35.841732 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:35:35.841676    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/368ca806-07df-48be-8ee6-3e8773114611-istio-data" (OuterVolumeSpecName: "istio-data") pod "368ca806-07df-48be-8ee6-3e8773114611" (UID: "368ca806-07df-48be-8ee6-3e8773114611"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:35:35.841732 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:35:35.841702    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/368ca806-07df-48be-8ee6-3e8773114611-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "368ca806-07df-48be-8ee6-3e8773114611" (UID: "368ca806-07df-48be-8ee6-3e8773114611"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:35:35.841999 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:35:35.841980    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/368ca806-07df-48be-8ee6-3e8773114611-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "368ca806-07df-48be-8ee6-3e8773114611" (UID: "368ca806-07df-48be-8ee6-3e8773114611"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:35:35.843391 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:35:35.843371    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/368ca806-07df-48be-8ee6-3e8773114611-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "368ca806-07df-48be-8ee6-3e8773114611" (UID: "368ca806-07df-48be-8ee6-3e8773114611"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:35:35.843572 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:35:35.843548    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/368ca806-07df-48be-8ee6-3e8773114611-istio-token" (OuterVolumeSpecName: "istio-token") pod "368ca806-07df-48be-8ee6-3e8773114611" (UID: "368ca806-07df-48be-8ee6-3e8773114611"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:35:35.843756 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:35:35.843737    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/368ca806-07df-48be-8ee6-3e8773114611-kube-api-access-7jrjb" (OuterVolumeSpecName: "kube-api-access-7jrjb") pod "368ca806-07df-48be-8ee6-3e8773114611" (UID: "368ca806-07df-48be-8ee6-3e8773114611"). InnerVolumeSpecName "kube-api-access-7jrjb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:35:35.843822 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:35:35.843749    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/368ca806-07df-48be-8ee6-3e8773114611-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "368ca806-07df-48be-8ee6-3e8773114611" (UID: "368ca806-07df-48be-8ee6-3e8773114611"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 14:35:35.943061 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:35:35.943027    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7jrjb\" (UniqueName: \"kubernetes.io/projected/368ca806-07df-48be-8ee6-3e8773114611-kube-api-access-7jrjb\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:35:35.943061 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:35:35.943058    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/368ca806-07df-48be-8ee6-3e8773114611-credential-socket\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:35:35.943264 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:35:35.943082    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/368ca806-07df-48be-8ee6-3e8773114611-istio-token\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:35:35.943264 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:35:35.943091    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/368ca806-07df-48be-8ee6-3e8773114611-workload-certs\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:35:35.943264 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:35:35.943099    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/368ca806-07df-48be-8ee6-3e8773114611-istio-data\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:35:35.943264 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:35:35.943107    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/368ca806-07df-48be-8ee6-3e8773114611-istio-envoy\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:35:35.943264 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:35:35.943114    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/368ca806-07df-48be-8ee6-3e8773114611-istio-podinfo\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:35:35.943264 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:35:35.943122    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/368ca806-07df-48be-8ee6-3e8773114611-istiod-ca-cert\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:35:36.682178 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:35:36.682144    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:35:36.682178 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:35:36.682156    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6" event={"ID":"368ca806-07df-48be-8ee6-3e8773114611","Type":"ContainerDied","Data":"d15ed72e9fb289407f344e7dd09ecd4a0441a82d585b4e8ff114b87593970f33"}
Sep 17 14:35:36.682715 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:35:36.682217    2582 scope.go:117] "RemoveContainer" containerID="b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09"
Sep 17 14:35:36.698711 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:35:36.698660    2582 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"368ca806-07df-48be-8ee6-3e8773114611\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"},\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"image\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"imageID\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"istio-proxy\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"limits\\\":{\\\"cpu\\\":\\\"2\\\",\\\"memory\\\":\\\"1Gi\\\"},\\\"requests\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"}},\\\"restartCount\\\":1,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-09-17T14:19:17Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":1000329999,\\\"supplementalGroups\\\":[1000329999,1000320000],\\\"uid\\\":1000329999}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-uds\\\",\\\"name\\\":\\\"workload-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/credential-uds\\\",\\\"name\\\":\\\"credential-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-credentials\\\",\\\"name\\\":\\\"workload-certs\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/istio\\\",\\\"name\\\":\\\"istiod-ca-cert\\\"},{\\\"mountPath\\\":\\\"/var/lib/istio/data\\\",\\\"name\\\":\\\"istio-data\\\"},{\\\"mountPath\\\":\\\"/etc/istio/proxy\\\",\\\"name\\\":\\\"istio-envoy\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/tokens\\\",\\\"name\\\":\\\"istio-token\\\"},{\\\"mountPath\\\":\\\"/etc/istio/pod\\\",\\\"name\\\":\\\"istio-podinfo\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-7jrjb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"observedGeneration\\\":2,\\\"phase\\\":\\\"Succeeded\\\"}}\" for pod \"openshift-ingress\"/\"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/pod?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:35:36.712793 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:35:36.712742    2582 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"368ca806-07df-48be-8ee6-3e8773114611\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"},\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"image\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"imageID\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"istio-proxy\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"limits\\\":{\\\"cpu\\\":\\\"2\\\",\\\"memory\\\":\\\"1Gi\\\"},\\\"requests\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"}},\\\"restartCount\\\":1,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-09-17T14:19:17Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":1000329999,\\\"supplementalGroups\\\":[1000329999,1000320000],\\\"uid\\\":1000329999}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-uds\\\",\\\"name\\\":\\\"workload-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/credential-uds\\\",\\\"name\\\":\\\"credential-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-credentials\\\",\\\"name\\\":\\\"workload-certs\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/istio\\\",\\\"name\\\":\\\"istiod-ca-cert\\\"},{\\\"mountPath\\\":\\\"/var/lib/istio/data\\\",\\\"name\\\":\\\"istio-data\\\"},{\\\"mountPath\\\":\\\"/etc/istio/proxy\\\",\\\"name\\\":\\\"istio-envoy\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/tokens\\\",\\\"name\\\":\\\"istio-token\\\"},{\\\"mountPath\\\":\\\"/etc/istio/pod\\\",\\\"name\\\":\\\"istio-podinfo\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-7jrjb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"observedGeneration\\\":2,\\\"phase\\\":\\\"Succeeded\\\"}}\" for pod \"openshift-ingress\"/\"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/pod?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:35:37.405251 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:35:37.405209    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:35:27Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:35:27Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:35:27Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:35:27Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Patch \"https://a2b89f3df576c40229a4e99950687ee2-0e02d3b4ed87b95e.elb.us-east-1.amazonaws.com:6443/api/v1/nodes/ip-10-0-137-53.ec2.internal/status?timeout=10s\": context deadline exceeded"
Sep 17 14:35:39.506482 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:35:39.506406    2582 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"368ca806-07df-48be-8ee6-3e8773114611\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"},\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"image\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"imageID\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"istio-proxy\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"limits\\\":{\\\"cpu\\\":\\\"2\\\",\\\"memory\\\":\\\"1Gi\\\"},\\\"requests\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"}},\\\"restartCount\\\":1,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-09-17T14:19:17Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":1000329999,\\\"supplementalGroups\\\":[1000329999,1000320000],\\\"uid\\\":1000329999}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-uds\\\",\\\"name\\\":\\\"workload-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/credential-uds\\\",\\\"name\\\":\\\"credential-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-credentials\\\",\\\"name\\\":\\\"workload-certs\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/istio\\\",\\\"name\\\":\\\"istiod-ca-cert\\\"},{\\\"mountPath\\\":\\\"/var/lib/istio/data\\\",\\\"name\\\":\\\"istio-data\\\"},{\\\"mountPath\\\":\\\"/etc/istio/proxy\\\",\\\"name\\\":\\\"istio-envoy\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/tokens\\\",\\\"name\\\":\\\"istio-token\\\"},{\\\"mountPath\\\":\\\"/etc/istio/pod\\\",\\\"name\\\":\\\"istio-podinfo\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-7jrjb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"observedGeneration\\\":2,\\\"phase\\\":\\\"Succeeded\\\"}}\" for pod \"openshift-ingress\"/\"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/pod?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:35:49.508390 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:35:49.508327    2582 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"368ca806-07df-48be-8ee6-3e8773114611\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"},\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"image\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"imageID\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"istio-proxy\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"limits\\\":{\\\"cpu\\\":\\\"2\\\",\\\"memory\\\":\\\"1Gi\\\"},\\\"requests\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"}},\\\"restartCount\\\":1,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-09-17T14:19:17Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":1000329999,\\\"supplementalGroups\\\":[1000329999,1000320000],\\\"uid\\\":1000329999}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-uds\\\",\\\"name\\\":\\\"workload-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/credential-uds\\\",\\\"name\\\":\\\"credential-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-credentials\\\",\\\"name\\\":\\\"workload-certs\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/istio\\\",\\\"name\\\":\\\"istiod-ca-cert\\\"},{\\\"mountPath\\\":\\\"/var/lib/istio/data\\\",\\\"name\\\":\\\"istio-data\\\"},{\\\"mountPath\\\":\\\"/etc/istio/proxy\\\",\\\"name\\\":\\\"istio-envoy\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/tokens\\\",\\\"name\\\":\\\"istio-token\\\"},{\\\"mountPath\\\":\\\"/etc/istio/pod\\\",\\\"name\\\":\\\"istio-podinfo\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-7jrjb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"observedGeneration\\\":2,\\\"phase\\\":\\\"Succeeded\\\"}}\" for pod \"openshift-ingress\"/\"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/pod?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:35:59.507294 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:35:59.507230    2582 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"368ca806-07df-48be-8ee6-3e8773114611\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"},\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"image\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"imageID\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"istio-proxy\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"limits\\\":{\\\"cpu\\\":\\\"2\\\",\\\"memory\\\":\\\"1Gi\\\"},\\\"requests\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"}},\\\"restartCount\\\":1,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-09-17T14:19:17Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":1000329999,\\\"supplementalGroups\\\":[1000329999,1000320000],\\\"uid\\\":1000329999}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-uds\\\",\\\"name\\\":\\\"workload-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/credential-uds\\\",\\\"name\\\":\\\"credential-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-credentials\\\",\\\"name\\\":\\\"workload-certs\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/istio\\\",\\\"name\\\":\\\"istiod-ca-cert\\\"},{\\\"mountPath\\\":\\\"/var/lib/istio/data\\\",\\\"name\\\":\\\"istio-data\\\"},{\\\"mountPath\\\":\\\"/etc/istio/proxy\\\",\\\"name\\\":\\\"istio-envoy\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/tokens\\\",\\\"name\\\":\\\"istio-token\\\"},{\\\"mountPath\\\":\\\"/etc/istio/pod\\\",\\\"name\\\":\\\"istio-podinfo\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-7jrjb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"observedGeneration\\\":2,\\\"phase\\\":\\\"Succeeded\\\"}}\" for pod \"openshift-ingress\"/\"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/pod?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:36:04.929434 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:36:04.929398    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:36:07.661543 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:36:07.661500    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:07Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:07Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:07Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:07Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:36:07.675202 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:36:07.675164    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:07Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:07Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:07Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:07Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:36:07.687422 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:36:07.687391    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:07Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:07Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:07Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:07Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:36:07.700760 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:36:07.700731    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:07Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:07Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:07Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:07Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:36:07.713447 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:36:07.713420    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:07Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:07Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:07Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:07Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:36:07.713447 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:36:07.713446    2582 kubelet_node_status.go:523] "Unable to update node status" err="update node status exceeds retry count"
Sep 17 14:36:09.505847 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:36:09.505787    2582 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"368ca806-07df-48be-8ee6-3e8773114611\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"},\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"image\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"imageID\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"istio-proxy\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"limits\\\":{\\\"cpu\\\":\\\"2\\\",\\\"memory\\\":\\\"1Gi\\\"},\\\"requests\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"}},\\\"restartCount\\\":1,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-09-17T14:19:17Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":1000329999,\\\"supplementalGroups\\\":[1000329999,1000320000],\\\"uid\\\":1000329999}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-uds\\\",\\\"name\\\":\\\"workload-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/credential-uds\\\",\\\"name\\\":\\\"credential-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-credentials\\\",\\\"name\\\":\\\"workload-certs\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/istio\\\",\\\"name\\\":\\\"istiod-ca-cert\\\"},{\\\"mountPath\\\":\\\"/var/lib/istio/data\\\",\\\"name\\\":\\\"istio-data\\\"},{\\\"mountPath\\\":\\\"/etc/istio/proxy\\\",\\\"name\\\":\\\"istio-envoy\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/tokens\\\",\\\"name\\\":\\\"istio-token\\\"},{\\\"mountPath\\\":\\\"/etc/istio/pod\\\",\\\"name\\\":\\\"istio-podinfo\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-7jrjb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"observedGeneration\\\":2,\\\"phase\\\":\\\"Succeeded\\\"}}\" for pod \"openshift-ingress\"/\"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/pod?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:36:19.507234 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:36:19.507166    2582 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"368ca806-07df-48be-8ee6-3e8773114611\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"},\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"image\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"imageID\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"istio-proxy\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"limits\\\":{\\\"cpu\\\":\\\"2\\\",\\\"memory\\\":\\\"1Gi\\\"},\\\"requests\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"}},\\\"restartCount\\\":1,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-09-17T14:19:17Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":1000329999,\\\"supplementalGroups\\\":[1000329999,1000320000],\\\"uid\\\":1000329999}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-uds\\\",\\\"name\\\":\\\"workload-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/credential-uds\\\",\\\"name\\\":\\\"credential-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-credentials\\\",\\\"name\\\":\\\"workload-certs\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/istio\\\",\\\"name\\\":\\\"istiod-ca-cert\\\"},{\\\"mountPath\\\":\\\"/var/lib/istio/data\\\",\\\"name\\\":\\\"istio-data\\\"},{\\\"mountPath\\\":\\\"/etc/istio/proxy\\\",\\\"name\\\":\\\"istio-envoy\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/tokens\\\",\\\"name\\\":\\\"istio-token\\\"},{\\\"mountPath\\\":\\\"/etc/istio/pod\\\",\\\"name\\\":\\\"istio-podinfo\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-7jrjb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"observedGeneration\\\":2,\\\"phase\\\":\\\"Succeeded\\\"}}\" for pod \"openshift-ingress\"/\"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/pod?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:36:29.507713 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:36:29.507591    2582 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"368ca806-07df-48be-8ee6-3e8773114611\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"},\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"image\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"imageID\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"istio-proxy\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"limits\\\":{\\\"cpu\\\":\\\"2\\\",\\\"memory\\\":\\\"1Gi\\\"},\\\"requests\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"}},\\\"restartCount\\\":1,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-09-17T14:19:17Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":1000329999,\\\"supplementalGroups\\\":[1000329999,1000320000],\\\"uid\\\":1000329999}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-uds\\\",\\\"name\\\":\\\"workload-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/credential-uds\\\",\\\"name\\\":\\\"credential-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-credentials\\\",\\\"name\\\":\\\"workload-certs\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/istio\\\",\\\"name\\\":\\\"istiod-ca-cert\\\"},{\\\"mountPath\\\":\\\"/var/lib/istio/data\\\",\\\"name\\\":\\\"istio-data\\\"},{\\\"mountPath\\\":\\\"/etc/istio/proxy\\\",\\\"name\\\":\\\"istio-envoy\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/tokens\\\",\\\"name\\\":\\\"istio-token\\\"},{\\\"mountPath\\\":\\\"/etc/istio/pod\\\",\\\"name\\\":\\\"istio-podinfo\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-7jrjb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"observedGeneration\\\":2,\\\"phase\\\":\\\"Succeeded\\\"}}\" for pod \"openshift-ingress\"/\"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/pod?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:36:38.086782 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:36:38.086731    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:38Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:38Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:38Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:38Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:36:38.105140 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:36:38.105092    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:38Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:38Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:38Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:38Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:36:38.118862 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:36:38.118804    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:38Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:38Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:38Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:38Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:36:38.132993 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:36:38.132952    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:38Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:38Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:38Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:38Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:36:38.147880 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:36:38.147842    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:38Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:38Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:38Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:38Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:36:38.147880 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:36:38.147878    2582 kubelet_node_status.go:523] "Unable to update node status" err="update node status exceeds retry count"
Sep 17 14:36:39.508274 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:36:39.508211    2582 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"368ca806-07df-48be-8ee6-3e8773114611\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"},\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"image\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"imageID\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"istio-proxy\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"limits\\\":{\\\"cpu\\\":\\\"2\\\",\\\"memory\\\":\\\"1Gi\\\"},\\\"requests\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"}},\\\"restartCount\\\":1,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-09-17T14:19:17Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":1000329999,\\\"supplementalGroups\\\":[1000329999,1000320000],\\\"uid\\\":1000329999}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-uds\\\",\\\"name\\\":\\\"workload-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/credential-uds\\\",\\\"name\\\":\\\"credential-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-credentials\\\",\\\"name\\\":\\\"workload-certs\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/istio\\\",\\\"name\\\":\\\"istiod-ca-cert\\\"},{\\\"mountPath\\\":\\\"/var/lib/istio/data\\\",\\\"name\\\":\\\"istio-data\\\"},{\\\"mountPath\\\":\\\"/etc/istio/proxy\\\",\\\"name\\\":\\\"istio-envoy\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/tokens\\\",\\\"name\\\":\\\"istio-token\\\"},{\\\"mountPath\\\":\\\"/etc/istio/pod\\\",\\\"name\\\":\\\"istio-podinfo\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-7jrjb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"observedGeneration\\\":2,\\\"phase\\\":\\\"Succeeded\\\"}}\" for pod \"openshift-ingress\"/\"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/pod?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:36:40.276624 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:36:40.276582    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:36:40.538801 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:36:40.538711    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:36:40.811326 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:36:40.811247    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:36:49.507833 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:36:49.507773    2582 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"368ca806-07df-48be-8ee6-3e8773114611\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"},\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"image\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"imageID\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"istio-proxy\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"limits\\\":{\\\"cpu\\\":\\\"2\\\",\\\"memory\\\":\\\"1Gi\\\"},\\\"requests\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"}},\\\"restartCount\\\":1,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-09-17T14:19:17Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":1000329999,\\\"supplementalGroups\\\":[1000329999,1000320000],\\\"uid\\\":1000329999}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-uds\\\",\\\"name\\\":\\\"workload-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/credential-uds\\\",\\\"name\\\":\\\"credential-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-credentials\\\",\\\"name\\\":\\\"workload-certs\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/istio\\\",\\\"name\\\":\\\"istiod-ca-cert\\\"},{\\\"mountPath\\\":\\\"/var/lib/istio/data\\\",\\\"name\\\":\\\"istio-data\\\"},{\\\"mountPath\\\":\\\"/etc/istio/proxy\\\",\\\"name\\\":\\\"istio-envoy\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/tokens\\\",\\\"name\\\":\\\"istio-token\\\"},{\\\"mountPath\\\":\\\"/etc/istio/pod\\\",\\\"name\\\":\\\"istio-podinfo\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-7jrjb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"observedGeneration\\\":2,\\\"phase\\\":\\\"Succeeded\\\"}}\" for pod \"openshift-ingress\"/\"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/pod?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:36:58.224490 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:36:58.224442    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:58Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:58Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:58Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:58Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:36:58.240367 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:36:58.240324    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:58Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:58Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:58Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:58Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:36:58.255532 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:36:58.255497    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:58Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:58Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:58Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:58Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:36:58.271381 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:36:58.271345    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:58Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:58Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:58Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:58Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:36:58.285223 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:36:58.285191    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:58Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:58Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:58Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:36:58Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:36:58.285223 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:36:58.285219    2582 kubelet_node_status.go:523] "Unable to update node status" err="update node status exceeds retry count"
Sep 17 14:36:59.510229 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:36:59.510146    2582 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"368ca806-07df-48be-8ee6-3e8773114611\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"},\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"image\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"imageID\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"istio-proxy\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"limits\\\":{\\\"cpu\\\":\\\"2\\\",\\\"memory\\\":\\\"1Gi\\\"},\\\"requests\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"}},\\\"restartCount\\\":1,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-09-17T14:19:17Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":1000329999,\\\"supplementalGroups\\\":[1000329999,1000320000],\\\"uid\\\":1000329999}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-uds\\\",\\\"name\\\":\\\"workload-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/credential-uds\\\",\\\"name\\\":\\\"credential-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-credentials\\\",\\\"name\\\":\\\"workload-certs\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/istio\\\",\\\"name\\\":\\\"istiod-ca-cert\\\"},{\\\"mountPath\\\":\\\"/var/lib/istio/data\\\",\\\"name\\\":\\\"istio-data\\\"},{\\\"mountPath\\\":\\\"/etc/istio/proxy\\\",\\\"name\\\":\\\"istio-envoy\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/tokens\\\",\\\"name\\\":\\\"istio-token\\\"},{\\\"mountPath\\\":\\\"/etc/istio/pod\\\",\\\"name\\\":\\\"istio-podinfo\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-7jrjb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"observedGeneration\\\":2,\\\"phase\\\":\\\"Succeeded\\\"}}\" for pod \"openshift-ingress\"/\"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/pod?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:36:59.923383 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:36:59.923297    2582 scope.go:117] "RemoveContainer" containerID="feb6891c586219001cf26b49cdecc61f408b6537254c288b98413779bf397e8f"
Sep 17 14:36:59.932446 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:36:59.932422    2582 scope.go:117] "RemoveContainer" containerID="a08616140d6acf8485d28366e2114b8095794d8bd32d5ff8280f06717da14ac2"
Sep 17 14:36:59.940672 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:36:59.940653    2582 scope.go:117] "RemoveContainer" containerID="f47d9fedc866c61d8316df21ebe9f7adb3bfd28469f925ec7af0ec5ccba812fd"
Sep 17 14:36:59.948837 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:36:59.948811    2582 scope.go:117] "RemoveContainer" containerID="29e47ec63ccc365eda39e8bbc6729b060249b6099a9c821190132119bfa5a67d"
Sep 17 14:36:59.956682 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:36:59.956659    2582 scope.go:117] "RemoveContainer" containerID="00571ddd2965add6034aeff80c1ad6bc986807d101d08e6c5b325334b569609e"
Sep 17 14:37:09.510626 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:37:09.510553    2582 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"368ca806-07df-48be-8ee6-3e8773114611\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"},\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"image\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"imageID\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"istio-proxy\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"limits\\\":{\\\"cpu\\\":\\\"2\\\",\\\"memory\\\":\\\"1Gi\\\"},\\\"requests\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"}},\\\"restartCount\\\":1,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-09-17T14:19:17Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":1000329999,\\\"supplementalGroups\\\":[1000329999,1000320000],\\\"uid\\\":1000329999}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-uds\\\",\\\"name\\\":\\\"workload-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/credential-uds\\\",\\\"name\\\":\\\"credential-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-credentials\\\",\\\"name\\\":\\\"workload-certs\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/istio\\\",\\\"name\\\":\\\"istiod-ca-cert\\\"},{\\\"mountPath\\\":\\\"/var/lib/istio/data\\\",\\\"name\\\":\\\"istio-data\\\"},{\\\"mountPath\\\":\\\"/etc/istio/proxy\\\",\\\"name\\\":\\\"istio-envoy\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/tokens\\\",\\\"name\\\":\\\"istio-token\\\"},{\\\"mountPath\\\":\\\"/etc/istio/pod\\\",\\\"name\\\":\\\"istio-podinfo\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-7jrjb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"observedGeneration\\\":2,\\\"phase\\\":\\\"Succeeded\\\"}}\" for pod \"openshift-ingress\"/\"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/pod?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:37:19.508558 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:37:19.508495    2582 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"368ca806-07df-48be-8ee6-3e8773114611\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"},\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"image\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"imageID\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"istio-proxy\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"limits\\\":{\\\"cpu\\\":\\\"2\\\",\\\"memory\\\":\\\"1Gi\\\"},\\\"requests\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"}},\\\"restartCount\\\":1,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-09-17T14:19:17Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":1000329999,\\\"supplementalGroups\\\":[1000329999,1000320000],\\\"uid\\\":1000329999}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-uds\\\",\\\"name\\\":\\\"workload-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/credential-uds\\\",\\\"name\\\":\\\"credential-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-credentials\\\",\\\"name\\\":\\\"workload-certs\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/istio\\\",\\\"name\\\":\\\"istiod-ca-cert\\\"},{\\\"mountPath\\\":\\\"/var/lib/istio/data\\\",\\\"name\\\":\\\"istio-data\\\"},{\\\"mountPath\\\":\\\"/etc/istio/proxy\\\",\\\"name\\\":\\\"istio-envoy\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/tokens\\\",\\\"name\\\":\\\"istio-token\\\"},{\\\"mountPath\\\":\\\"/etc/istio/pod\\\",\\\"name\\\":\\\"istio-podinfo\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-7jrjb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"observedGeneration\\\":2,\\\"phase\\\":\\\"Succeeded\\\"}}\" for pod \"openshift-ingress\"/\"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/pod?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:37:28.727125 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:37:28.727077    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:37:28Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:37:28Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:37:28Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:37:28Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:37:28.743243 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:37:28.743202    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:37:28Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:37:28Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:37:28Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:37:28Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:37:28.758663 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:37:28.758624    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:37:28Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:37:28Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:37:28Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:37:28Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:37:28.774588 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:37:28.774548    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:37:28Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:37:28Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:37:28Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:37:28Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:37:28.789278 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:37:28.789237    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:37:28Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:37:28Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:37:28Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:37:28Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:37:28.789278 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:37:28.789276    2582 kubelet_node_status.go:523] "Unable to update node status" err="update node status exceeds retry count"
Sep 17 14:37:29.508319 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:37:29.508235    2582 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"368ca806-07df-48be-8ee6-3e8773114611\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"},\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"image\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"imageID\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"istio-proxy\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"limits\\\":{\\\"cpu\\\":\\\"2\\\",\\\"memory\\\":\\\"1Gi\\\"},\\\"requests\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"}},\\\"restartCount\\\":1,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-09-17T14:19:17Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":1000329999,\\\"supplementalGroups\\\":[1000329999,1000320000],\\\"uid\\\":1000329999}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-uds\\\",\\\"name\\\":\\\"workload-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/credential-uds\\\",\\\"name\\\":\\\"credential-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-credentials\\\",\\\"name\\\":\\\"workload-certs\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/istio\\\",\\\"name\\\":\\\"istiod-ca-cert\\\"},{\\\"mountPath\\\":\\\"/var/lib/istio/data\\\",\\\"name\\\":\\\"istio-data\\\"},{\\\"mountPath\\\":\\\"/etc/istio/proxy\\\",\\\"name\\\":\\\"istio-envoy\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/tokens\\\",\\\"name\\\":\\\"istio-token\\\"},{\\\"mountPath\\\":\\\"/etc/istio/pod\\\",\\\"name\\\":\\\"istio-podinfo\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-7jrjb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"observedGeneration\\\":2,\\\"phase\\\":\\\"Succeeded\\\"}}\" for pod \"openshift-ingress\"/\"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/pod?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:37:38.887416 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:37:38.887380    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:37:38Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:37:38Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:37:38Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:37:38Z\\\",\\\"type\\\":\\\"Ready\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:37:39.506350 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:37:39.506246    2582 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"368ca806-07df-48be-8ee6-3e8773114611\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"},\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"image\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"imageID\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"istio-proxy\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"limits\\\":{\\\"cpu\\\":\\\"2\\\",\\\"memory\\\":\\\"1Gi\\\"},\\\"requests\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"}},\\\"restartCount\\\":1,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-09-17T14:19:17Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":1000329999,\\\"supplementalGroups\\\":[1000329999,1000320000],\\\"uid\\\":1000329999}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-uds\\\",\\\"name\\\":\\\"workload-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/credential-uds\\\",\\\"name\\\":\\\"credential-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-credentials\\\",\\\"name\\\":\\\"workload-certs\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/istio\\\",\\\"name\\\":\\\"istiod-ca-cert\\\"},{\\\"mountPath\\\":\\\"/var/lib/istio/data\\\",\\\"name\\\":\\\"istio-data\\\"},{\\\"mountPath\\\":\\\"/etc/istio/proxy\\\",\\\"name\\\":\\\"istio-envoy\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/tokens\\\",\\\"name\\\":\\\"istio-token\\\"},{\\\"mountPath\\\":\\\"/etc/istio/pod\\\",\\\"name\\\":\\\"istio-podinfo\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-7jrjb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"observedGeneration\\\":2,\\\"phase\\\":\\\"Succeeded\\\"}}\" for pod \"openshift-ingress\"/\"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/pod?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:37:49.510714 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:37:49.510651    2582 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"368ca806-07df-48be-8ee6-3e8773114611\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"},\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"image\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"imageID\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"istio-proxy\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"limits\\\":{\\\"cpu\\\":\\\"2\\\",\\\"memory\\\":\\\"1Gi\\\"},\\\"requests\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"}},\\\"restartCount\\\":1,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-09-17T14:19:17Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":1000329999,\\\"supplementalGroups\\\":[1000329999,1000320000],\\\"uid\\\":1000329999}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-uds\\\",\\\"name\\\":\\\"workload-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/credential-uds\\\",\\\"name\\\":\\\"credential-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-credentials\\\",\\\"name\\\":\\\"workload-certs\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/istio\\\",\\\"name\\\":\\\"istiod-ca-cert\\\"},{\\\"mountPath\\\":\\\"/var/lib/istio/data\\\",\\\"name\\\":\\\"istio-data\\\"},{\\\"mountPath\\\":\\\"/etc/istio/proxy\\\",\\\"name\\\":\\\"istio-envoy\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/tokens\\\",\\\"name\\\":\\\"istio-token\\\"},{\\\"mountPath\\\":\\\"/etc/istio/pod\\\",\\\"name\\\":\\\"istio-podinfo\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-7jrjb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"observedGeneration\\\":2,\\\"phase\\\":\\\"Succeeded\\\"}}\" for pod \"openshift-ingress\"/\"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/pod?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:37:59.507334 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:37:59.507275    2582 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"368ca806-07df-48be-8ee6-3e8773114611\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"},\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"image\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"imageID\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"istio-proxy\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"limits\\\":{\\\"cpu\\\":\\\"2\\\",\\\"memory\\\":\\\"1Gi\\\"},\\\"requests\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"}},\\\"restartCount\\\":1,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-09-17T14:19:17Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":1000329999,\\\"supplementalGroups\\\":[1000329999,1000320000],\\\"uid\\\":1000329999}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-uds\\\",\\\"name\\\":\\\"workload-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/credential-uds\\\",\\\"name\\\":\\\"credential-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-credentials\\\",\\\"name\\\":\\\"workload-certs\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/istio\\\",\\\"name\\\":\\\"istiod-ca-cert\\\"},{\\\"mountPath\\\":\\\"/var/lib/istio/data\\\",\\\"name\\\":\\\"istio-data\\\"},{\\\"mountPath\\\":\\\"/etc/istio/proxy\\\",\\\"name\\\":\\\"istio-envoy\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/tokens\\\",\\\"name\\\":\\\"istio-token\\\"},{\\\"mountPath\\\":\\\"/etc/istio/pod\\\",\\\"name\\\":\\\"istio-podinfo\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-7jrjb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"observedGeneration\\\":2,\\\"phase\\\":\\\"Succeeded\\\"}}\" for pod \"openshift-ingress\"/\"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/pod?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:37:59.512491 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:37:59.512462    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:37:59Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:37:59Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:37:59Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:37:59Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:37:59.528337 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:37:59.528293    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:37:59Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:37:59Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:37:59Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:37:59Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:37:59.544836 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:37:59.544805    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:37:59Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:37:59Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:37:59Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:37:59Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:37:59.561139 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:37:59.561102    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:37:59Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:37:59Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:37:59Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:37:59Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:37:59.574481 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:37:59.574446    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:37:59Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:37:59Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:37:59Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:37:59Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:37:59.574481 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:37:59.574475    2582 kubelet_node_status.go:523] "Unable to update node status" err="update node status exceeds retry count"
Sep 17 14:37:59.993555 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:37:59.993523    2582 scope.go:117] "RemoveContainer" containerID="c27bfc8db786168ee575fc8a8f383ca1f07878f684db337d887d697db4abc159"
Sep 17 14:38:00.002059 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:38:00.002041    2582 scope.go:117] "RemoveContainer" containerID="609b7b8a94a6a490036df07c976910830321dac4fd76e6fb31c4f7db16748d4d"
Sep 17 14:38:00.009869 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:38:00.009849    2582 scope.go:117] "RemoveContainer" containerID="e5ef38dd92a69a63826f54271d93d0bf195620a93a5431c1aa89b9844a8afea9"
Sep 17 14:38:09.508885 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:38:09.508824    2582 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"368ca806-07df-48be-8ee6-3e8773114611\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"},\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"image\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"imageID\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"istio-proxy\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"limits\\\":{\\\"cpu\\\":\\\"2\\\",\\\"memory\\\":\\\"1Gi\\\"},\\\"requests\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"}},\\\"restartCount\\\":1,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-09-17T14:19:17Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":1000329999,\\\"supplementalGroups\\\":[1000329999,1000320000],\\\"uid\\\":1000329999}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-uds\\\",\\\"name\\\":\\\"workload-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/credential-uds\\\",\\\"name\\\":\\\"credential-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-credentials\\\",\\\"name\\\":\\\"workload-certs\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/istio\\\",\\\"name\\\":\\\"istiod-ca-cert\\\"},{\\\"mountPath\\\":\\\"/var/lib/istio/data\\\",\\\"name\\\":\\\"istio-data\\\"},{\\\"mountPath\\\":\\\"/etc/istio/proxy\\\",\\\"name\\\":\\\"istio-envoy\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/tokens\\\",\\\"name\\\":\\\"istio-token\\\"},{\\\"mountPath\\\":\\\"/etc/istio/pod\\\",\\\"name\\\":\\\"istio-podinfo\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-7jrjb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"observedGeneration\\\":2,\\\"phase\\\":\\\"Succeeded\\\"}}\" for pod \"openshift-ingress\"/\"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/pod?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:38:19.509093 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:38:19.509033    2582 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"368ca806-07df-48be-8ee6-3e8773114611\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"},\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"image\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"imageID\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"istio-proxy\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"limits\\\":{\\\"cpu\\\":\\\"2\\\",\\\"memory\\\":\\\"1Gi\\\"},\\\"requests\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"}},\\\"restartCount\\\":1,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-09-17T14:19:17Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":1000329999,\\\"supplementalGroups\\\":[1000329999,1000320000],\\\"uid\\\":1000329999}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-uds\\\",\\\"name\\\":\\\"workload-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/credential-uds\\\",\\\"name\\\":\\\"credential-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-credentials\\\",\\\"name\\\":\\\"workload-certs\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/istio\\\",\\\"name\\\":\\\"istiod-ca-cert\\\"},{\\\"mountPath\\\":\\\"/var/lib/istio/data\\\",\\\"name\\\":\\\"istio-data\\\"},{\\\"mountPath\\\":\\\"/etc/istio/proxy\\\",\\\"name\\\":\\\"istio-envoy\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/tokens\\\",\\\"name\\\":\\\"istio-token\\\"},{\\\"mountPath\\\":\\\"/etc/istio/pod\\\",\\\"name\\\":\\\"istio-podinfo\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-7jrjb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"observedGeneration\\\":2,\\\"phase\\\":\\\"Succeeded\\\"}}\" for pod \"openshift-ingress\"/\"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/pod?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:38:29.513949 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:38:29.513888    2582 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"368ca806-07df-48be-8ee6-3e8773114611\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"},\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"image\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"imageID\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"istio-proxy\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"limits\\\":{\\\"cpu\\\":\\\"2\\\",\\\"memory\\\":\\\"1Gi\\\"},\\\"requests\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"}},\\\"restartCount\\\":1,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-09-17T14:19:17Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":1000329999,\\\"supplementalGroups\\\":[1000329999,1000320000],\\\"uid\\\":1000329999}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-uds\\\",\\\"name\\\":\\\"workload-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/credential-uds\\\",\\\"name\\\":\\\"credential-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-credentials\\\",\\\"name\\\":\\\"workload-certs\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/istio\\\",\\\"name\\\":\\\"istiod-ca-cert\\\"},{\\\"mountPath\\\":\\\"/var/lib/istio/data\\\",\\\"name\\\":\\\"istio-data\\\"},{\\\"mountPath\\\":\\\"/etc/istio/proxy\\\",\\\"name\\\":\\\"istio-envoy\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/tokens\\\",\\\"name\\\":\\\"istio-token\\\"},{\\\"mountPath\\\":\\\"/etc/istio/pod\\\",\\\"name\\\":\\\"istio-podinfo\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-7jrjb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"observedGeneration\\\":2,\\\"phase\\\":\\\"Succeeded\\\"}}\" for pod \"openshift-ingress\"/\"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/pod?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:38:39.507012 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:38:39.506949    2582 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"368ca806-07df-48be-8ee6-3e8773114611\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"},\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"image\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"imageID\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"istio-proxy\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"limits\\\":{\\\"cpu\\\":\\\"2\\\",\\\"memory\\\":\\\"1Gi\\\"},\\\"requests\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"}},\\\"restartCount\\\":1,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-09-17T14:19:17Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":1000329999,\\\"supplementalGroups\\\":[1000329999,1000320000],\\\"uid\\\":1000329999}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-uds\\\",\\\"name\\\":\\\"workload-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/credential-uds\\\",\\\"name\\\":\\\"credential-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-credentials\\\",\\\"name\\\":\\\"workload-certs\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/istio\\\",\\\"name\\\":\\\"istiod-ca-cert\\\"},{\\\"mountPath\\\":\\\"/var/lib/istio/data\\\",\\\"name\\\":\\\"istio-data\\\"},{\\\"mountPath\\\":\\\"/etc/istio/proxy\\\",\\\"name\\\":\\\"istio-envoy\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/tokens\\\",\\\"name\\\":\\\"istio-token\\\"},{\\\"mountPath\\\":\\\"/etc/istio/pod\\\",\\\"name\\\":\\\"istio-podinfo\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-7jrjb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"observedGeneration\\\":2,\\\"phase\\\":\\\"Succeeded\\\"}}\" for pod \"openshift-ingress\"/\"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/pod?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:38:49.515467 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:38:49.515404    2582 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"368ca806-07df-48be-8ee6-3e8773114611\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"},\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"image\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"imageID\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"istio-proxy\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"limits\\\":{\\\"cpu\\\":\\\"2\\\",\\\"memory\\\":\\\"1Gi\\\"},\\\"requests\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"}},\\\"restartCount\\\":1,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-09-17T14:19:17Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":1000329999,\\\"supplementalGroups\\\":[1000329999,1000320000],\\\"uid\\\":1000329999}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-uds\\\",\\\"name\\\":\\\"workload-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/credential-uds\\\",\\\"name\\\":\\\"credential-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-credentials\\\",\\\"name\\\":\\\"workload-certs\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/istio\\\",\\\"name\\\":\\\"istiod-ca-cert\\\"},{\\\"mountPath\\\":\\\"/var/lib/istio/data\\\",\\\"name\\\":\\\"istio-data\\\"},{\\\"mountPath\\\":\\\"/etc/istio/proxy\\\",\\\"name\\\":\\\"istio-envoy\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/tokens\\\",\\\"name\\\":\\\"istio-token\\\"},{\\\"mountPath\\\":\\\"/etc/istio/pod\\\",\\\"name\\\":\\\"istio-podinfo\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-7jrjb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"observedGeneration\\\":2,\\\"phase\\\":\\\"Succeeded\\\"}}\" for pod \"openshift-ingress\"/\"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/pod?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:38:50.513280 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:38:50.513231    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:38:50Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:38:50Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:38:50Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:38:50Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:38:50.528594 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:38:50.528552    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:38:50Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:38:50Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:38:50Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:38:50Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:38:50.544427 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:38:50.544384    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:38:50Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:38:50Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:38:50Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:38:50Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:38:50.560655 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:38:50.560601    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:38:50Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:38:50Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:38:50Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:38:50Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:38:50.574599 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:38:50.574556    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:38:50Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:38:50Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:38:50Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:38:50Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:38:50.574599 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:38:50.574592    2582 kubelet_node_status.go:523] "Unable to update node status" err="update node status exceeds retry count"
Sep 17 14:38:59.509175 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:38:59.509112    2582 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"368ca806-07df-48be-8ee6-3e8773114611\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"},\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"image\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"imageID\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"istio-proxy\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"limits\\\":{\\\"cpu\\\":\\\"2\\\",\\\"memory\\\":\\\"1Gi\\\"},\\\"requests\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"}},\\\"restartCount\\\":1,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-09-17T14:19:17Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":1000329999,\\\"supplementalGroups\\\":[1000329999,1000320000],\\\"uid\\\":1000329999}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-uds\\\",\\\"name\\\":\\\"workload-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/credential-uds\\\",\\\"name\\\":\\\"credential-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-credentials\\\",\\\"name\\\":\\\"workload-certs\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/istio\\\",\\\"name\\\":\\\"istiod-ca-cert\\\"},{\\\"mountPath\\\":\\\"/var/lib/istio/data\\\",\\\"name\\\":\\\"istio-data\\\"},{\\\"mountPath\\\":\\\"/etc/istio/proxy\\\",\\\"name\\\":\\\"istio-envoy\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/tokens\\\",\\\"name\\\":\\\"istio-token\\\"},{\\\"mountPath\\\":\\\"/etc/istio/pod\\\",\\\"name\\\":\\\"istio-podinfo\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-7jrjb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"observedGeneration\\\":2,\\\"phase\\\":\\\"Succeeded\\\"}}\" for pod \"openshift-ingress\"/\"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/pod?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:39:00.042814 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:39:00.042780    2582 scope.go:117] "RemoveContainer" containerID="2d661b208c2934ede0a6f9f1ba0872340820efce48b0eb2a71d57a7e3c598b74"
Sep 17 14:39:00.051351 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:39:00.051324    2582 scope.go:117] "RemoveContainer" containerID="56c60a036bc64a6f336dc768825ddee926ff31b252513e31cc7f12d5178b56bc"
Sep 17 14:39:00.923277 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:39:00.923239    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:00Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:00Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:00Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:00Z\\\",\\\"type\\\":\\\"Ready\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:39:09.509913 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:39:09.509846    2582 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"368ca806-07df-48be-8ee6-3e8773114611\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"},\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"image\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"imageID\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"istio-proxy\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"limits\\\":{\\\"cpu\\\":\\\"2\\\",\\\"memory\\\":\\\"1Gi\\\"},\\\"requests\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"}},\\\"restartCount\\\":1,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-09-17T14:19:17Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":1000329999,\\\"supplementalGroups\\\":[1000329999,1000320000],\\\"uid\\\":1000329999}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-uds\\\",\\\"name\\\":\\\"workload-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/credential-uds\\\",\\\"name\\\":\\\"credential-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-credentials\\\",\\\"name\\\":\\\"workload-certs\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/istio\\\",\\\"name\\\":\\\"istiod-ca-cert\\\"},{\\\"mountPath\\\":\\\"/var/lib/istio/data\\\",\\\"name\\\":\\\"istio-data\\\"},{\\\"mountPath\\\":\\\"/etc/istio/proxy\\\",\\\"name\\\":\\\"istio-envoy\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/tokens\\\",\\\"name\\\":\\\"istio-token\\\"},{\\\"mountPath\\\":\\\"/etc/istio/pod\\\",\\\"name\\\":\\\"istio-podinfo\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-7jrjb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"observedGeneration\\\":2,\\\"phase\\\":\\\"Succeeded\\\"}}\" for pod \"openshift-ingress\"/\"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/pod?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:39:11.221652 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:39:11.221590    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:11Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:11Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:11Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:11Z\\\",\\\"type\\\":\\\"Ready\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:39:11.237197 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:39:11.237159    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:11Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:11Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:11Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:11Z\\\",\\\"type\\\":\\\"Ready\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:39:11.253815 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:39:11.253769    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:11Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:11Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:11Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:11Z\\\",\\\"type\\\":\\\"Ready\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:39:11.268198 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:39:11.268154    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:11Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:11Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:11Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:11Z\\\",\\\"type\\\":\\\"Ready\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:39:11.282272 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:39:11.282231    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:11Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:11Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:11Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:11Z\\\",\\\"type\\\":\\\"Ready\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:39:11.282272 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:39:11.282262    2582 kubelet_node_status.go:523] "Unable to update node status" err="update node status exceeds retry count"
Sep 17 14:39:19.509424 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:39:19.509355    2582 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"368ca806-07df-48be-8ee6-3e8773114611\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"},\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"image\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"imageID\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"istio-proxy\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"limits\\\":{\\\"cpu\\\":\\\"2\\\",\\\"memory\\\":\\\"1Gi\\\"},\\\"requests\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"}},\\\"restartCount\\\":1,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-09-17T14:19:17Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":1000329999,\\\"supplementalGroups\\\":[1000329999,1000320000],\\\"uid\\\":1000329999}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-uds\\\",\\\"name\\\":\\\"workload-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/credential-uds\\\",\\\"name\\\":\\\"credential-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-credentials\\\",\\\"name\\\":\\\"workload-certs\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/istio\\\",\\\"name\\\":\\\"istiod-ca-cert\\\"},{\\\"mountPath\\\":\\\"/var/lib/istio/data\\\",\\\"name\\\":\\\"istio-data\\\"},{\\\"mountPath\\\":\\\"/etc/istio/proxy\\\",\\\"name\\\":\\\"istio-envoy\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/tokens\\\",\\\"name\\\":\\\"istio-token\\\"},{\\\"mountPath\\\":\\\"/etc/istio/pod\\\",\\\"name\\\":\\\"istio-podinfo\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-7jrjb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"observedGeneration\\\":2,\\\"phase\\\":\\\"Succeeded\\\"}}\" for pod \"openshift-ingress\"/\"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/pod?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:39:21.354029 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:39:21.353991    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:21Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:21Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:21Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:21Z\\\",\\\"type\\\":\\\"Ready\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:39:21.369513 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:39:21.369468    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:21Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:21Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:21Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:21Z\\\",\\\"type\\\":\\\"Ready\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:39:21.384187 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:39:21.384155    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:21Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:21Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:21Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:21Z\\\",\\\"type\\\":\\\"Ready\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:39:21.400045 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:39:21.400012    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:21Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:21Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:21Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:21Z\\\",\\\"type\\\":\\\"Ready\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:39:21.415288 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:39:21.415255    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:21Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:21Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:21Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:21Z\\\",\\\"type\\\":\\\"Ready\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:39:21.415288 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:39:21.415283    2582 kubelet_node_status.go:523] "Unable to update node status" err="update node status exceeds retry count"
Sep 17 14:39:29.510529 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:39:29.510418    2582 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"368ca806-07df-48be-8ee6-3e8773114611\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"},\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"image\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"imageID\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"istio-proxy\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"limits\\\":{\\\"cpu\\\":\\\"2\\\",\\\"memory\\\":\\\"1Gi\\\"},\\\"requests\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"}},\\\"restartCount\\\":1,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-09-17T14:19:17Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":1000329999,\\\"supplementalGroups\\\":[1000329999,1000320000],\\\"uid\\\":1000329999}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-uds\\\",\\\"name\\\":\\\"workload-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/credential-uds\\\",\\\"name\\\":\\\"credential-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-credentials\\\",\\\"name\\\":\\\"workload-certs\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/istio\\\",\\\"name\\\":\\\"istiod-ca-cert\\\"},{\\\"mountPath\\\":\\\"/var/lib/istio/data\\\",\\\"name\\\":\\\"istio-data\\\"},{\\\"mountPath\\\":\\\"/etc/istio/proxy\\\",\\\"name\\\":\\\"istio-envoy\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/tokens\\\",\\\"name\\\":\\\"istio-token\\\"},{\\\"mountPath\\\":\\\"/etc/istio/pod\\\",\\\"name\\\":\\\"istio-podinfo\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-7jrjb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"observedGeneration\\\":2,\\\"phase\\\":\\\"Succeeded\\\"}}\" for pod \"openshift-ingress\"/\"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/pod?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:39:31.605095 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:39:31.605048    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:31Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:31Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:31Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:31Z\\\",\\\"type\\\":\\\"Ready\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:39:31.690441 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:39:31.690396    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:31Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:31Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:31Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:31Z\\\",\\\"type\\\":\\\"Ready\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:39:31.717446 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:39:31.717407    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:31Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:31Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:31Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:31Z\\\",\\\"type\\\":\\\"Ready\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:39:31.813829 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:39:31.813784    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:31Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:31Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:31Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:31Z\\\",\\\"type\\\":\\\"Ready\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:39:31.926449 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:39:31.926351    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:31Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:31Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:31Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:31Z\\\",\\\"type\\\":\\\"Ready\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:39:31.926449 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:39:31.926392    2582 kubelet_node_status.go:523] "Unable to update node status" err="update node status exceeds retry count"
Sep 17 14:39:39.536181 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:39:39.536097    2582 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"368ca806-07df-48be-8ee6-3e8773114611\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"},\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"image\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"imageID\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"istio-proxy\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"limits\\\":{\\\"cpu\\\":\\\"2\\\",\\\"memory\\\":\\\"1Gi\\\"},\\\"requests\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"}},\\\"restartCount\\\":1,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-09-17T14:19:17Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":1000329999,\\\"supplementalGroups\\\":[1000329999,1000320000],\\\"uid\\\":1000329999}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-uds\\\",\\\"name\\\":\\\"workload-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/credential-uds\\\",\\\"name\\\":\\\"credential-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-credentials\\\",\\\"name\\\":\\\"workload-certs\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/istio\\\",\\\"name\\\":\\\"istiod-ca-cert\\\"},{\\\"mountPath\\\":\\\"/var/lib/istio/data\\\",\\\"name\\\":\\\"istio-data\\\"},{\\\"mountPath\\\":\\\"/etc/istio/proxy\\\",\\\"name\\\":\\\"istio-envoy\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/tokens\\\",\\\"name\\\":\\\"istio-token\\\"},{\\\"mountPath\\\":\\\"/etc/istio/pod\\\",\\\"name\\\":\\\"istio-podinfo\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-7jrjb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"observedGeneration\\\":2,\\\"phase\\\":\\\"Succeeded\\\"}}\" for pod \"openshift-ingress\"/\"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/pod?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:39:42.258405 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:39:42.258361    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:42Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:42Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:42Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:42Z\\\",\\\"type\\\":\\\"Ready\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:39:42.277850 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:39:42.277803    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:42Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:42Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:42Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:42Z\\\",\\\"type\\\":\\\"Ready\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:39:42.305691 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:39:42.305654    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:42Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:42Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:42Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:42Z\\\",\\\"type\\\":\\\"Ready\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:39:42.330116 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:39:42.330074    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:42Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:42Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:42Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:42Z\\\",\\\"type\\\":\\\"Ready\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:39:42.352323 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:39:42.352284    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:42Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:42Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:42Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:42Z\\\",\\\"type\\\":\\\"Ready\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:39:42.352323 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:39:42.352316    2582 kubelet_node_status.go:523] "Unable to update node status" err="update node status exceeds retry count"
Sep 17 14:39:49.510122 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:39:49.510052    2582 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"368ca806-07df-48be-8ee6-3e8773114611\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"},\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"image\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"imageID\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"istio-proxy\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"limits\\\":{\\\"cpu\\\":\\\"2\\\",\\\"memory\\\":\\\"1Gi\\\"},\\\"requests\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"}},\\\"restartCount\\\":1,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-09-17T14:19:17Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":1000329999,\\\"supplementalGroups\\\":[1000329999,1000320000],\\\"uid\\\":1000329999}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-uds\\\",\\\"name\\\":\\\"workload-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/credential-uds\\\",\\\"name\\\":\\\"credential-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-credentials\\\",\\\"name\\\":\\\"workload-certs\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/istio\\\",\\\"name\\\":\\\"istiod-ca-cert\\\"},{\\\"mountPath\\\":\\\"/var/lib/istio/data\\\",\\\"name\\\":\\\"istio-data\\\"},{\\\"mountPath\\\":\\\"/etc/istio/proxy\\\",\\\"name\\\":\\\"istio-envoy\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/tokens\\\",\\\"name\\\":\\\"istio-token\\\"},{\\\"mountPath\\\":\\\"/etc/istio/pod\\\",\\\"name\\\":\\\"istio-podinfo\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-7jrjb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"observedGeneration\\\":2,\\\"phase\\\":\\\"Succeeded\\\"}}\" for pod \"openshift-ingress\"/\"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/pod?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:39:52.627372 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:39:52.627335    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:52Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:52Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:52Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:52Z\\\",\\\"type\\\":\\\"Ready\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:39:52.666420 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:39:52.666374    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:52Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:52Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:52Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:52Z\\\",\\\"type\\\":\\\"Ready\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:39:52.684207 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:39:52.684169    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:52Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:52Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:52Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:52Z\\\",\\\"type\\\":\\\"Ready\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:39:52.700483 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:39:52.700443    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:52Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:52Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:52Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:52Z\\\",\\\"type\\\":\\\"Ready\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:39:52.717402 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:39:52.717365    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:52Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:52Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:52Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:39:52Z\\\",\\\"type\\\":\\\"Ready\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:39:52.717402 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:39:52.717393    2582 kubelet_node_status.go:523] "Unable to update node status" err="update node status exceeds retry count"
Sep 17 14:39:59.507047 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:39:59.506960    2582 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"368ca806-07df-48be-8ee6-3e8773114611\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"},\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"image\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"imageID\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"istio-proxy\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"limits\\\":{\\\"cpu\\\":\\\"2\\\",\\\"memory\\\":\\\"1Gi\\\"},\\\"requests\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"}},\\\"restartCount\\\":1,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://b3b94a91dfd3f656dc5d8ae8bdbb29ed8e8e36ba9baabec79db15528d7986c09\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-09-17T14:35:35Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-09-17T14:19:17Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":1000329999,\\\"supplementalGroups\\\":[1000329999,1000320000],\\\"uid\\\":1000329999}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-uds\\\",\\\"name\\\":\\\"workload-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/credential-uds\\\",\\\"name\\\":\\\"credential-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-credentials\\\",\\\"name\\\":\\\"workload-certs\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/istio\\\",\\\"name\\\":\\\"istiod-ca-cert\\\"},{\\\"mountPath\\\":\\\"/var/lib/istio/data\\\",\\\"name\\\":\\\"istio-data\\\"},{\\\"mountPath\\\":\\\"/etc/istio/proxy\\\",\\\"name\\\":\\\"istio-envoy\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/tokens\\\",\\\"name\\\":\\\"istio-token\\\"},{\\\"mountPath\\\":\\\"/etc/istio/pod\\\",\\\"name\\\":\\\"istio-podinfo\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-7jrjb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"observedGeneration\\\":2,\\\"phase\\\":\\\"Succeeded\\\"}}\" for pod \"openshift-ingress\"/\"e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/pod?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused" pod="openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"
Sep 17 14:39:59.644476 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:39:59.644440    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-rr97k_45ffa13d-635a-46e8-8894-30902b9cf719/console-operator/2.log"
Sep 17 14:39:59.651314 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:39:59.651290    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-rr97k_45ffa13d-635a-46e8-8894-30902b9cf719/console-operator/2.log"
Sep 17 14:40:02.859786 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:40:02.859743    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:40:02Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:40:02Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:40:02Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:40:02Z\\\",\\\"type\\\":\\\"Ready\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:40:02.878712 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:40:02.878671    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:40:02Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:40:02Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:40:02Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:40:02Z\\\",\\\"type\\\":\\\"Ready\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:40:02.896291 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:40:02.896249    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:40:02Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:40:02Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:40:02Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:40:02Z\\\",\\\"type\\\":\\\"Ready\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:40:02.918258 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:40:02.918207    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:40:02Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:40:02Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:40:02Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:40:02Z\\\",\\\"type\\\":\\\"Ready\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:40:02.941249 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:40:02.941203    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:40:02Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:40:02Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:40:02Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-17T14:40:02Z\\\",\\\"type\\\":\\\"Ready\\\"}]}}\" for node \"ip-10-0-137-53.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-6f4d9a9c-b466-48fc-bb29-13cef4544b0c.svc:9743/node?timeout=10s\": dial tcp 172.30.162.63:9743: connect: connection refused"
Sep 17 14:40:02.941249 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:40:02.941239    2582 kubelet_node_status.go:523] "Unable to update node status" err="update node status exceeds retry count"
Sep 17 14:40:09.551979 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:40:09.551942    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"]
Sep 17 14:40:09.565679 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:40:09.565645    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-fcd9f3-openshift-default-d4fcdc8f4-zlqx6"]
Sep 17 14:40:11.493984 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:40:11.493936    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="368ca806-07df-48be-8ee6-3e8773114611" path="/var/lib/kubelet/pods/368ca806-07df-48be-8ee6-3e8773114611/volumes"
Sep 17 14:42:15.711257 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:15.711216    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:42:19.454918 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:19.454873    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:42:43.940494 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:43.940409    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-cc3801-g7smh"]
Sep 17 14:42:43.940877 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:43.940843    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3731cd4a-3fe7-4a2c-8d28-bda2472f1042" containerName="main"
Sep 17 14:42:43.940877 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:43.940856    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3731cd4a-3fe7-4a2c-8d28-bda2472f1042" containerName="main"
Sep 17 14:42:43.940877 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:43.940875    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="775a4af7-36fe-429a-9fde-d85bc8ce5cfb" containerName="main"
Sep 17 14:42:43.940987 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:43.940881    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="775a4af7-36fe-429a-9fde-d85bc8ce5cfb" containerName="main"
Sep 17 14:42:43.940987 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:43.940891    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d2f89a0-b1b2-47d9-976a-2435cf66750a" containerName="main"
Sep 17 14:42:43.940987 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:43.940897    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d2f89a0-b1b2-47d9-976a-2435cf66750a" containerName="main"
Sep 17 14:42:43.940987 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:43.940903    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a035404c-dad3-4c44-9d8a-d816be9f0550" containerName="revoke-keys"
Sep 17 14:42:43.940987 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:43.940909    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a035404c-dad3-4c44-9d8a-d816be9f0550" containerName="revoke-keys"
Sep 17 14:42:43.940987 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:43.940929    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="368ca806-07df-48be-8ee6-3e8773114611" containerName="istio-proxy"
Sep 17 14:42:43.940987 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:43.940934    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="368ca806-07df-48be-8ee6-3e8773114611" containerName="istio-proxy"
Sep 17 14:42:43.940987 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:43.940943    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="368ca806-07df-48be-8ee6-3e8773114611" containerName="istio-proxy"
Sep 17 14:42:43.940987 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:43.940948    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="368ca806-07df-48be-8ee6-3e8773114611" containerName="istio-proxy"
Sep 17 14:42:43.941301 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:43.941013    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="68dd2b1c-ed84-4291-8fb7-f61779fd5b85" containerName="istio-proxy"
Sep 17 14:42:43.941301 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:43.941020    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="775a4af7-36fe-429a-9fde-d85bc8ce5cfb" containerName="main"
Sep 17 14:42:43.941301 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:43.941027    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3731cd4a-3fe7-4a2c-8d28-bda2472f1042" containerName="main"
Sep 17 14:42:43.941301 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:43.941034    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a035404c-dad3-4c44-9d8a-d816be9f0550" containerName="revoke-keys"
Sep 17 14:42:43.941301 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:43.941041    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="368ca806-07df-48be-8ee6-3e8773114611" containerName="istio-proxy"
Sep 17 14:42:43.941301 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:43.941046    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="368ca806-07df-48be-8ee6-3e8773114611" containerName="istio-proxy"
Sep 17 14:42:43.941301 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:43.941053    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d2f89a0-b1b2-47d9-976a-2435cf66750a" containerName="main"
Sep 17 14:42:43.944023 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:43.944002    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-cc3801-g7smh"
Sep 17 14:42:43.946828 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:43.946801    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ddwrn\""
Sep 17 14:42:43.965263 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:43.965231    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-cc3801-g7smh"]
Sep 17 14:42:43.999217 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:43.999179    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08c395f3-3214-418b-a122-7a3afa6b2363-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-cc3801-g7smh\" (UID: \"08c395f3-3214-418b-a122-7a3afa6b2363\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-cc3801-g7smh"
Sep 17 14:42:44.100678 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:44.100601    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08c395f3-3214-418b-a122-7a3afa6b2363-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-cc3801-g7smh\" (UID: \"08c395f3-3214-418b-a122-7a3afa6b2363\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-cc3801-g7smh"
Sep 17 14:42:44.101307 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:44.101281    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08c395f3-3214-418b-a122-7a3afa6b2363-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-cc3801-g7smh\" (UID: \"08c395f3-3214-418b-a122-7a3afa6b2363\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-cc3801-g7smh"
Sep 17 14:42:44.254593 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:44.254553    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-cc3801-g7smh"
Sep 17 14:42:44.527714 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:44.527683    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-cc3801-g7smh"]
Sep 17 14:42:44.530263 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:42:44.530236    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08c395f3_3214_418b_a122_7a3afa6b2363.slice/crio-1c655a37df1734bd45d223aec2ae9f436e78167242eb5438a95467c8b6c52bf8 WatchSource:0}: Error finding container 1c655a37df1734bd45d223aec2ae9f436e78167242eb5438a95467c8b6c52bf8: Status 404 returned error can't find the container with id 1c655a37df1734bd45d223aec2ae9f436e78167242eb5438a95467c8b6c52bf8
Sep 17 14:42:45.068757 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:45.068712    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-cc3801-g7smh" event={"ID":"08c395f3-3214-418b-a122-7a3afa6b2363","Type":"ContainerStarted","Data":"21886ed6b21e90a492e098b91537e197a93f381ca75235b752e5bb31ad0b2393"}
Sep 17 14:42:45.069191 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:45.068777    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-cc3801-g7smh" event={"ID":"08c395f3-3214-418b-a122-7a3afa6b2363","Type":"ContainerStarted","Data":"1c655a37df1734bd45d223aec2ae9f436e78167242eb5438a95467c8b6c52bf8"}
Sep 17 14:42:45.086886 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:45.086763    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-cc3801-g7smh" podStartSLOduration=15.086744137 podStartE2EDuration="15.086744137s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:42:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:42:45.086454142 +0000 UTC m=+2266.087997617" watchObservedRunningTime="2026-09-17 14:42:45.086744137 +0000 UTC m=+2266.088287613"
Sep 17 14:42:46.076351 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:46.076317    2582 generic.go:356] "Generic (PLEG): container finished" podID="08c395f3-3214-418b-a122-7a3afa6b2363" containerID="21886ed6b21e90a492e098b91537e197a93f381ca75235b752e5bb31ad0b2393" exitCode=0
Sep 17 14:42:46.076830 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:46.076394    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-cc3801-g7smh" event={"ID":"08c395f3-3214-418b-a122-7a3afa6b2363","Type":"ContainerDied","Data":"21886ed6b21e90a492e098b91537e197a93f381ca75235b752e5bb31ad0b2393"}
Sep 17 14:42:47.217709 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:47.217684    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-cc3801-g7smh"
Sep 17 14:42:47.341137 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:47.341033    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08c395f3-3214-418b-a122-7a3afa6b2363-maas-api-service-ca\") pod \"08c395f3-3214-418b-a122-7a3afa6b2363\" (UID: \"08c395f3-3214-418b-a122-7a3afa6b2363\") "
Sep 17 14:42:47.341412 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:47.341386    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08c395f3-3214-418b-a122-7a3afa6b2363-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "08c395f3-3214-418b-a122-7a3afa6b2363" (UID: "08c395f3-3214-418b-a122-7a3afa6b2363"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:42:47.442132 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:47.442077    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08c395f3-3214-418b-a122-7a3afa6b2363-maas-api-service-ca\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:42:48.088174 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:48.088145    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-cc3801-g7smh"
Sep 17 14:42:48.088174 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:48.088160    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-cc3801-g7smh" event={"ID":"08c395f3-3214-418b-a122-7a3afa6b2363","Type":"ContainerDied","Data":"1c655a37df1734bd45d223aec2ae9f436e78167242eb5438a95467c8b6c52bf8"}
Sep 17 14:42:48.088431 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:48.088202    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1c655a37df1734bd45d223aec2ae9f436e78167242eb5438a95467c8b6c52bf8"
Sep 17 14:42:50.690963 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:50.690917    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:42:51.033384 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:51.033347    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:42:51.072990 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:51.072957    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-cc3801-29827590-96rhs"]
Sep 17 14:42:51.076876 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:51.076842    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-cc3801-29827590-96rhs"]
Sep 17 14:42:51.175529 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:51.175492    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:42:51.312769 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:51.312678    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-cc3801-g7smh"]
Sep 17 14:42:51.317678 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:51.317647    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-cc3801-g7smh"]
Sep 17 14:42:51.503856 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:51.503821    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08c395f3-3214-418b-a122-7a3afa6b2363" path="/var/lib/kubelet/pods/08c395f3-3214-418b-a122-7a3afa6b2363/volumes"
Sep 17 14:42:51.504141 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:51.504128    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6d9f887-eea1-488c-abc4-01981a9e8724" path="/var/lib/kubelet/pods/f6d9f887-eea1-488c-abc4-01981a9e8724/volumes"
Sep 17 14:42:51.627351 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:51.627254    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:42:51.763792 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:51.763748    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:42:51.963243 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:51.963204    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:42:52.765200 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:52.765160    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:42:53.404072 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:53.404034    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:42:53.505831 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:53.505791    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:42:55.988232 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:42:55.988200    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:43:00.101023 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:00.100992    2582 scope.go:117] "RemoveContainer" containerID="3e3f50fb650f05876336c9cac251291c20a3b6c268170fa5765899c2a6c7b17b"
Sep 17 14:43:04.975453 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:04.975414    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:43:33.387343 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:33.387296    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:43:33.486749 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:33.486710    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:43:33.648054 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:33.647956    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:43:39.429279 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:39.429212    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:43:39.528702 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:39.528664    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:43:39.712256 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:39.712221    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr"]
Sep 17 14:43:39.712512 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:39.712472    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr" podUID="7d7bc84f-6345-4a14-a73b-21e0aa0a30ff" containerName="istio-proxy" containerID="cri-o://b1ce9c406d84163c001f8f324b4a6d479e408f829427cfc48b88efef00170171" gracePeriod=30
Sep 17 14:43:39.739712 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:39.739666    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:43:40.127101 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:40.127012    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4c29a2-68bd7"]
Sep 17 14:43:40.127713 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:40.127694    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08c395f3-3214-418b-a122-7a3afa6b2363" containerName="revoke-keys"
Sep 17 14:43:40.127770 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:40.127716    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="08c395f3-3214-418b-a122-7a3afa6b2363" containerName="revoke-keys"
Sep 17 14:43:40.127827 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:40.127816    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="08c395f3-3214-418b-a122-7a3afa6b2363" containerName="revoke-keys"
Sep 17 14:43:40.131379 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:40.131359    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4c29a2-68bd7"
Sep 17 14:43:40.134016 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:40.133992    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ddwrn\""
Sep 17 14:43:40.138024 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:40.137996    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4c29a2-68bd7"]
Sep 17 14:43:40.260408 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:40.260370    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d078ec98-8e40-4f83-9fd8-6646f383eb16-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-4c29a2-68bd7\" (UID: \"d078ec98-8e40-4f83-9fd8-6646f383eb16\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4c29a2-68bd7"
Sep 17 14:43:40.361296 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:40.361258    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d078ec98-8e40-4f83-9fd8-6646f383eb16-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-4c29a2-68bd7\" (UID: \"d078ec98-8e40-4f83-9fd8-6646f383eb16\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4c29a2-68bd7"
Sep 17 14:43:40.361968 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:40.361945    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d078ec98-8e40-4f83-9fd8-6646f383eb16-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-4c29a2-68bd7\" (UID: \"d078ec98-8e40-4f83-9fd8-6646f383eb16\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4c29a2-68bd7"
Sep 17 14:43:40.443798 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:40.443764    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4c29a2-68bd7"
Sep 17 14:43:40.721066 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:40.721028    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4c29a2-68bd7"]
Sep 17 14:43:40.722480 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:43:40.722449    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd078ec98_8e40_4f83_9fd8_6646f383eb16.slice/crio-97b199fec5ea559c96ec0e2f5d2b17d26d3be7f754abfade138101fc7a6b9d18 WatchSource:0}: Error finding container 97b199fec5ea559c96ec0e2f5d2b17d26d3be7f754abfade138101fc7a6b9d18: Status 404 returned error can't find the container with id 97b199fec5ea559c96ec0e2f5d2b17d26d3be7f754abfade138101fc7a6b9d18
Sep 17 14:43:41.257559 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:41.257522    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:43:41.377909 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:41.377824    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4c29a2-68bd7" event={"ID":"d078ec98-8e40-4f83-9fd8-6646f383eb16","Type":"ContainerStarted","Data":"4b57f8d1184f880815f0c3a257d0ca834076f55ae1823068e858eff8fed1651a"}
Sep 17 14:43:41.377909 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:41.377882    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4c29a2-68bd7" event={"ID":"d078ec98-8e40-4f83-9fd8-6646f383eb16","Type":"ContainerStarted","Data":"97b199fec5ea559c96ec0e2f5d2b17d26d3be7f754abfade138101fc7a6b9d18"}
Sep 17 14:43:41.397403 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:41.397358    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4c29a2-68bd7" podStartSLOduration=1.397343464 podStartE2EDuration="1.397343464s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:43:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:43:41.397174109 +0000 UTC m=+2322.398717583" watchObservedRunningTime="2026-09-17 14:43:41.397343464 +0000 UTC m=+2322.398886939"
Sep 17 14:43:41.982625 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:41.982572    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:43:42.385002 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:42.384912    2582 generic.go:356] "Generic (PLEG): container finished" podID="d078ec98-8e40-4f83-9fd8-6646f383eb16" containerID="4b57f8d1184f880815f0c3a257d0ca834076f55ae1823068e858eff8fed1651a" exitCode=0
Sep 17 14:43:42.385174 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:42.384999    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4c29a2-68bd7" event={"ID":"d078ec98-8e40-4f83-9fd8-6646f383eb16","Type":"ContainerDied","Data":"4b57f8d1184f880815f0c3a257d0ca834076f55ae1823068e858eff8fed1651a"}
Sep 17 14:43:43.530602 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:43.530576    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4c29a2-68bd7"
Sep 17 14:43:43.589566 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:43.589533    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d078ec98-8e40-4f83-9fd8-6646f383eb16-maas-api-service-ca\") pod \"d078ec98-8e40-4f83-9fd8-6646f383eb16\" (UID: \"d078ec98-8e40-4f83-9fd8-6646f383eb16\") "
Sep 17 14:43:43.589885 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:43.589862    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d078ec98-8e40-4f83-9fd8-6646f383eb16-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d078ec98-8e40-4f83-9fd8-6646f383eb16" (UID: "d078ec98-8e40-4f83-9fd8-6646f383eb16"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:43:43.691196 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:43.691104    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d078ec98-8e40-4f83-9fd8-6646f383eb16-maas-api-service-ca\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:43:44.396775 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:44.396742    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4c29a2-68bd7"
Sep 17 14:43:44.396775 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:44.396769    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4c29a2-68bd7" event={"ID":"d078ec98-8e40-4f83-9fd8-6646f383eb16","Type":"ContainerDied","Data":"97b199fec5ea559c96ec0e2f5d2b17d26d3be7f754abfade138101fc7a6b9d18"}
Sep 17 14:43:44.396992 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:44.396814    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="97b199fec5ea559c96ec0e2f5d2b17d26d3be7f754abfade138101fc7a6b9d18"
Sep 17 14:43:44.970318 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:44.970260    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr"
Sep 17 14:43:45.004964 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:45.004924    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wwxt9\" (UniqueName: \"kubernetes.io/projected/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-kube-api-access-wwxt9\") pod \"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff\" (UID: \"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff\") "
Sep 17 14:43:45.004964 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:45.004970    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-istiod-ca-cert\") pod \"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff\" (UID: \"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff\") "
Sep 17 14:43:45.005213 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:45.005011    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-istio-envoy\") pod \"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff\" (UID: \"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff\") "
Sep 17 14:43:45.005213 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:45.005042    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-workload-certs\") pod \"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff\" (UID: \"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff\") "
Sep 17 14:43:45.005213 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:45.005106    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-istio-data\") pod \"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff\" (UID: \"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff\") "
Sep 17 14:43:45.005213 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:45.005144    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-credential-socket\") pod \"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff\" (UID: \"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff\") "
Sep 17 14:43:45.005213 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:45.005177    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-istio-token\") pod \"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff\" (UID: \"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff\") "
Sep 17 14:43:45.005489 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:45.005216    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-workload-socket\") pod \"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff\" (UID: \"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff\") "
Sep 17 14:43:45.005489 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:45.005257    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-istio-podinfo\") pod \"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff\" (UID: \"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff\") "
Sep 17 14:43:45.005489 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:45.005370    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7d7bc84f-6345-4a14-a73b-21e0aa0a30ff" (UID: "7d7bc84f-6345-4a14-a73b-21e0aa0a30ff"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:43:45.005489 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:45.005479    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7d7bc84f-6345-4a14-a73b-21e0aa0a30ff" (UID: "7d7bc84f-6345-4a14-a73b-21e0aa0a30ff"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:43:45.005732 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:45.005509    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7d7bc84f-6345-4a14-a73b-21e0aa0a30ff" (UID: "7d7bc84f-6345-4a14-a73b-21e0aa0a30ff"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:43:45.005732 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:45.005693    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-istio-data" (OuterVolumeSpecName: "istio-data") pod "7d7bc84f-6345-4a14-a73b-21e0aa0a30ff" (UID: "7d7bc84f-6345-4a14-a73b-21e0aa0a30ff"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:43:45.005845 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:45.005742    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-credential-socket\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:43:45.005845 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:45.005762    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-istiod-ca-cert\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:43:45.005845 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:45.005776    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-workload-certs\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:43:45.005996 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:45.005882    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7d7bc84f-6345-4a14-a73b-21e0aa0a30ff" (UID: "7d7bc84f-6345-4a14-a73b-21e0aa0a30ff"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:43:45.007503 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:45.007469    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7d7bc84f-6345-4a14-a73b-21e0aa0a30ff" (UID: "7d7bc84f-6345-4a14-a73b-21e0aa0a30ff"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:43:45.007723 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:45.007701    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-istio-token" (OuterVolumeSpecName: "istio-token") pod "7d7bc84f-6345-4a14-a73b-21e0aa0a30ff" (UID: "7d7bc84f-6345-4a14-a73b-21e0aa0a30ff"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:43:45.007919 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:45.007895    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7d7bc84f-6345-4a14-a73b-21e0aa0a30ff" (UID: "7d7bc84f-6345-4a14-a73b-21e0aa0a30ff"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 14:43:45.007919 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:45.007911    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-kube-api-access-wwxt9" (OuterVolumeSpecName: "kube-api-access-wwxt9") pod "7d7bc84f-6345-4a14-a73b-21e0aa0a30ff" (UID: "7d7bc84f-6345-4a14-a73b-21e0aa0a30ff"). InnerVolumeSpecName "kube-api-access-wwxt9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:43:45.107264 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:45.107225    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-istio-envoy\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:43:45.107264 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:45.107257    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-istio-data\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:43:45.107264 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:45.107269    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-istio-token\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:43:45.107515 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:45.107277    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-workload-socket\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:43:45.107515 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:45.107285    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-istio-podinfo\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:43:45.107515 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:45.107294    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wwxt9\" (UniqueName: \"kubernetes.io/projected/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff-kube-api-access-wwxt9\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:43:45.403946 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:45.403852    2582 generic.go:356] "Generic (PLEG): container finished" podID="7d7bc84f-6345-4a14-a73b-21e0aa0a30ff" containerID="b1ce9c406d84163c001f8f324b4a6d479e408f829427cfc48b88efef00170171" exitCode=0
Sep 17 14:43:45.403946 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:45.403911    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr"
Sep 17 14:43:45.404132 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:45.403912    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr" event={"ID":"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff","Type":"ContainerDied","Data":"b1ce9c406d84163c001f8f324b4a6d479e408f829427cfc48b88efef00170171"}
Sep 17 14:43:45.404132 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:45.404028    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr" event={"ID":"7d7bc84f-6345-4a14-a73b-21e0aa0a30ff","Type":"ContainerDied","Data":"97a109c91dd86c242577443266931fb6845ec87cf349473c1eb4686462c6ecf0"}
Sep 17 14:43:45.404132 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:45.404058    2582 scope.go:117] "RemoveContainer" containerID="b1ce9c406d84163c001f8f324b4a6d479e408f829427cfc48b88efef00170171"
Sep 17 14:43:45.414040 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:45.414015    2582 scope.go:117] "RemoveContainer" containerID="a4a20a5a971d62d9f1307dbdfd286435147255ae51792d1c0388e3ca5efbfd12"
Sep 17 14:43:45.422582 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:45.422563    2582 scope.go:117] "RemoveContainer" containerID="b1ce9c406d84163c001f8f324b4a6d479e408f829427cfc48b88efef00170171"
Sep 17 14:43:45.422853 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:43:45.422833    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1ce9c406d84163c001f8f324b4a6d479e408f829427cfc48b88efef00170171\": container with ID starting with b1ce9c406d84163c001f8f324b4a6d479e408f829427cfc48b88efef00170171 not found: ID does not exist" containerID="b1ce9c406d84163c001f8f324b4a6d479e408f829427cfc48b88efef00170171"
Sep 17 14:43:45.422914 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:45.422863    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1ce9c406d84163c001f8f324b4a6d479e408f829427cfc48b88efef00170171"} err="failed to get container status \"b1ce9c406d84163c001f8f324b4a6d479e408f829427cfc48b88efef00170171\": rpc error: code = NotFound desc = could not find container \"b1ce9c406d84163c001f8f324b4a6d479e408f829427cfc48b88efef00170171\": container with ID starting with b1ce9c406d84163c001f8f324b4a6d479e408f829427cfc48b88efef00170171 not found: ID does not exist"
Sep 17 14:43:45.422914 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:45.422880    2582 scope.go:117] "RemoveContainer" containerID="a4a20a5a971d62d9f1307dbdfd286435147255ae51792d1c0388e3ca5efbfd12"
Sep 17 14:43:45.423131 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:43:45.423112    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4a20a5a971d62d9f1307dbdfd286435147255ae51792d1c0388e3ca5efbfd12\": container with ID starting with a4a20a5a971d62d9f1307dbdfd286435147255ae51792d1c0388e3ca5efbfd12 not found: ID does not exist" containerID="a4a20a5a971d62d9f1307dbdfd286435147255ae51792d1c0388e3ca5efbfd12"
Sep 17 14:43:45.423181 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:45.423137    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4a20a5a971d62d9f1307dbdfd286435147255ae51792d1c0388e3ca5efbfd12"} err="failed to get container status \"a4a20a5a971d62d9f1307dbdfd286435147255ae51792d1c0388e3ca5efbfd12\": rpc error: code = NotFound desc = could not find container \"a4a20a5a971d62d9f1307dbdfd286435147255ae51792d1c0388e3ca5efbfd12\": container with ID starting with a4a20a5a971d62d9f1307dbdfd286435147255ae51792d1c0388e3ca5efbfd12 not found: ID does not exist"
Sep 17 14:43:45.428726 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:45.428700    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr"]
Sep 17 14:43:45.436720 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:45.436698    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-cc3801-openshift-default-56c68b8d4d-82qfr"]
Sep 17 14:43:45.493679 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:45.493654    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d7bc84f-6345-4a14-a73b-21e0aa0a30ff" path="/var/lib/kubelet/pods/7d7bc84f-6345-4a14-a73b-21e0aa0a30ff/volumes"
Sep 17 14:43:50.789081 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:50.789040    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:43:50.990313 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:50.990274    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:43:51.003256 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:51.003216    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4c29a2-68bd7"]
Sep 17 14:43:51.007107 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:51.007080    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-4c29a2-68bd7"]
Sep 17 14:43:51.065875 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:51.065798    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:43:51.415367 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:51.415273    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:43:51.485898 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:51.485860    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:43:51.496651 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:51.496596    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d078ec98-8e40-4f83-9fd8-6646f383eb16" path="/var/lib/kubelet/pods/d078ec98-8e40-4f83-9fd8-6646f383eb16/volumes"
Sep 17 14:43:57.553411 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:57.553377    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:43:57.628055 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:57.628013    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:43:57.823593 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:57.823501    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd"]
Sep 17 14:43:57.823819 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:57.823791    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd" podUID="9d882585-6a65-4e36-b1c7-7ec4562cd182" containerName="istio-proxy" containerID="cri-o://31c892e1a703191ff331e65bb3aee00153a112911ffb915aa9bc0efe0513cc9a" gracePeriod=30
Sep 17 14:43:57.855406 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:43:57.855355    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:44:03.084003 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:44:03.083976    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd"
Sep 17 14:44:03.177982 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:44:03.177892    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d882585-6a65-4e36-b1c7-7ec4562cd182-credential-socket\") pod \"9d882585-6a65-4e36-b1c7-7ec4562cd182\" (UID: \"9d882585-6a65-4e36-b1c7-7ec4562cd182\") "
Sep 17 14:44:03.177982 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:44:03.177947    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dvbz8\" (UniqueName: \"kubernetes.io/projected/9d882585-6a65-4e36-b1c7-7ec4562cd182-kube-api-access-dvbz8\") pod \"9d882585-6a65-4e36-b1c7-7ec4562cd182\" (UID: \"9d882585-6a65-4e36-b1c7-7ec4562cd182\") "
Sep 17 14:44:03.177982 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:44:03.177970    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d882585-6a65-4e36-b1c7-7ec4562cd182-workload-socket\") pod \"9d882585-6a65-4e36-b1c7-7ec4562cd182\" (UID: \"9d882585-6a65-4e36-b1c7-7ec4562cd182\") "
Sep 17 14:44:03.178279 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:44:03.178072    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d882585-6a65-4e36-b1c7-7ec4562cd182-istio-data\") pod \"9d882585-6a65-4e36-b1c7-7ec4562cd182\" (UID: \"9d882585-6a65-4e36-b1c7-7ec4562cd182\") "
Sep 17 14:44:03.178279 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:44:03.178127    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d882585-6a65-4e36-b1c7-7ec4562cd182-istiod-ca-cert\") pod \"9d882585-6a65-4e36-b1c7-7ec4562cd182\" (UID: \"9d882585-6a65-4e36-b1c7-7ec4562cd182\") "
Sep 17 14:44:03.178279 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:44:03.178185    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d882585-6a65-4e36-b1c7-7ec4562cd182-workload-certs\") pod \"9d882585-6a65-4e36-b1c7-7ec4562cd182\" (UID: \"9d882585-6a65-4e36-b1c7-7ec4562cd182\") "
Sep 17 14:44:03.178279 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:44:03.178235    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d882585-6a65-4e36-b1c7-7ec4562cd182-istio-envoy\") pod \"9d882585-6a65-4e36-b1c7-7ec4562cd182\" (UID: \"9d882585-6a65-4e36-b1c7-7ec4562cd182\") "
Sep 17 14:44:03.178279 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:44:03.178254    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d882585-6a65-4e36-b1c7-7ec4562cd182-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9d882585-6a65-4e36-b1c7-7ec4562cd182" (UID: "9d882585-6a65-4e36-b1c7-7ec4562cd182"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:44:03.178536 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:44:03.178281    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d882585-6a65-4e36-b1c7-7ec4562cd182-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9d882585-6a65-4e36-b1c7-7ec4562cd182" (UID: "9d882585-6a65-4e36-b1c7-7ec4562cd182"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:44:03.178536 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:44:03.178291    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d882585-6a65-4e36-b1c7-7ec4562cd182-istio-podinfo\") pod \"9d882585-6a65-4e36-b1c7-7ec4562cd182\" (UID: \"9d882585-6a65-4e36-b1c7-7ec4562cd182\") "
Sep 17 14:44:03.178536 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:44:03.178363    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d882585-6a65-4e36-b1c7-7ec4562cd182-istio-data" (OuterVolumeSpecName: "istio-data") pod "9d882585-6a65-4e36-b1c7-7ec4562cd182" (UID: "9d882585-6a65-4e36-b1c7-7ec4562cd182"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:44:03.178536 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:44:03.178381    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d882585-6a65-4e36-b1c7-7ec4562cd182-istio-token\") pod \"9d882585-6a65-4e36-b1c7-7ec4562cd182\" (UID: \"9d882585-6a65-4e36-b1c7-7ec4562cd182\") "
Sep 17 14:44:03.178536 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:44:03.178522    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d882585-6a65-4e36-b1c7-7ec4562cd182-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9d882585-6a65-4e36-b1c7-7ec4562cd182" (UID: "9d882585-6a65-4e36-b1c7-7ec4562cd182"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:44:03.178831 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:44:03.178568    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d882585-6a65-4e36-b1c7-7ec4562cd182-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9d882585-6a65-4e36-b1c7-7ec4562cd182" (UID: "9d882585-6a65-4e36-b1c7-7ec4562cd182"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:44:03.178831 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:44:03.178781    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d882585-6a65-4e36-b1c7-7ec4562cd182-credential-socket\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:44:03.178831 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:44:03.178804    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d882585-6a65-4e36-b1c7-7ec4562cd182-workload-socket\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:44:03.178831 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:44:03.178819    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d882585-6a65-4e36-b1c7-7ec4562cd182-istio-data\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:44:03.178831 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:44:03.178832    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d882585-6a65-4e36-b1c7-7ec4562cd182-istiod-ca-cert\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:44:03.179022 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:44:03.178844    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d882585-6a65-4e36-b1c7-7ec4562cd182-workload-certs\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:44:03.180757 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:44:03.180730    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d882585-6a65-4e36-b1c7-7ec4562cd182-kube-api-access-dvbz8" (OuterVolumeSpecName: "kube-api-access-dvbz8") pod "9d882585-6a65-4e36-b1c7-7ec4562cd182" (UID: "9d882585-6a65-4e36-b1c7-7ec4562cd182"). InnerVolumeSpecName "kube-api-access-dvbz8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:44:03.180879 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:44:03.180760    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9d882585-6a65-4e36-b1c7-7ec4562cd182-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9d882585-6a65-4e36-b1c7-7ec4562cd182" (UID: "9d882585-6a65-4e36-b1c7-7ec4562cd182"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 14:44:03.180879 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:44:03.180843    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d882585-6a65-4e36-b1c7-7ec4562cd182-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9d882585-6a65-4e36-b1c7-7ec4562cd182" (UID: "9d882585-6a65-4e36-b1c7-7ec4562cd182"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:44:03.180968 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:44:03.180890    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d882585-6a65-4e36-b1c7-7ec4562cd182-istio-token" (OuterVolumeSpecName: "istio-token") pod "9d882585-6a65-4e36-b1c7-7ec4562cd182" (UID: "9d882585-6a65-4e36-b1c7-7ec4562cd182"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:44:03.279751 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:44:03.279721    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d882585-6a65-4e36-b1c7-7ec4562cd182-istio-envoy\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:44:03.279751 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:44:03.279748    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d882585-6a65-4e36-b1c7-7ec4562cd182-istio-podinfo\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:44:03.279751 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:44:03.279757    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d882585-6a65-4e36-b1c7-7ec4562cd182-istio-token\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:44:03.280004 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:44:03.279769    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dvbz8\" (UniqueName: \"kubernetes.io/projected/9d882585-6a65-4e36-b1c7-7ec4562cd182-kube-api-access-dvbz8\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:44:03.500873 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:44:03.500831    2582 generic.go:356] "Generic (PLEG): container finished" podID="9d882585-6a65-4e36-b1c7-7ec4562cd182" containerID="31c892e1a703191ff331e65bb3aee00153a112911ffb915aa9bc0efe0513cc9a" exitCode=0
Sep 17 14:44:03.500873 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:44:03.500872    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd"
Sep 17 14:44:03.501282 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:44:03.500901    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd" event={"ID":"9d882585-6a65-4e36-b1c7-7ec4562cd182","Type":"ContainerDied","Data":"31c892e1a703191ff331e65bb3aee00153a112911ffb915aa9bc0efe0513cc9a"}
Sep 17 14:44:03.501282 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:44:03.500949    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd" event={"ID":"9d882585-6a65-4e36-b1c7-7ec4562cd182","Type":"ContainerDied","Data":"a247dd8efdf7c88c056642a1388ffc0b47bae78628043b44f034978d83dbe99b"}
Sep 17 14:44:03.501282 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:44:03.500972    2582 scope.go:117] "RemoveContainer" containerID="31c892e1a703191ff331e65bb3aee00153a112911ffb915aa9bc0efe0513cc9a"
Sep 17 14:44:03.510469 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:44:03.510442    2582 scope.go:117] "RemoveContainer" containerID="c9cb969d5b97ef6679223ed8a0aa15f66d5d9d381a8d3b5b7d14a25c9252516e"
Sep 17 14:44:03.519645 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:44:03.519619    2582 scope.go:117] "RemoveContainer" containerID="31c892e1a703191ff331e65bb3aee00153a112911ffb915aa9bc0efe0513cc9a"
Sep 17 14:44:03.519776 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:44:03.519753    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd"]
Sep 17 14:44:03.520001 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:44:03.519973    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31c892e1a703191ff331e65bb3aee00153a112911ffb915aa9bc0efe0513cc9a\": container with ID starting with 31c892e1a703191ff331e65bb3aee00153a112911ffb915aa9bc0efe0513cc9a not found: ID does not exist" containerID="31c892e1a703191ff331e65bb3aee00153a112911ffb915aa9bc0efe0513cc9a"
Sep 17 14:44:03.520049 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:44:03.520010    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31c892e1a703191ff331e65bb3aee00153a112911ffb915aa9bc0efe0513cc9a"} err="failed to get container status \"31c892e1a703191ff331e65bb3aee00153a112911ffb915aa9bc0efe0513cc9a\": rpc error: code = NotFound desc = could not find container \"31c892e1a703191ff331e65bb3aee00153a112911ffb915aa9bc0efe0513cc9a\": container with ID starting with 31c892e1a703191ff331e65bb3aee00153a112911ffb915aa9bc0efe0513cc9a not found: ID does not exist"
Sep 17 14:44:03.520049 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:44:03.520028    2582 scope.go:117] "RemoveContainer" containerID="c9cb969d5b97ef6679223ed8a0aa15f66d5d9d381a8d3b5b7d14a25c9252516e"
Sep 17 14:44:03.520286 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:44:03.520270    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9cb969d5b97ef6679223ed8a0aa15f66d5d9d381a8d3b5b7d14a25c9252516e\": container with ID starting with c9cb969d5b97ef6679223ed8a0aa15f66d5d9d381a8d3b5b7d14a25c9252516e not found: ID does not exist" containerID="c9cb969d5b97ef6679223ed8a0aa15f66d5d9d381a8d3b5b7d14a25c9252516e"
Sep 17 14:44:03.520340 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:44:03.520290    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9cb969d5b97ef6679223ed8a0aa15f66d5d9d381a8d3b5b7d14a25c9252516e"} err="failed to get container status \"c9cb969d5b97ef6679223ed8a0aa15f66d5d9d381a8d3b5b7d14a25c9252516e\": rpc error: code = NotFound desc = could not find container \"c9cb969d5b97ef6679223ed8a0aa15f66d5d9d381a8d3b5b7d14a25c9252516e\": container with ID starting with c9cb969d5b97ef6679223ed8a0aa15f66d5d9d381a8d3b5b7d14a25c9252516e not found: ID does not exist"
Sep 17 14:44:03.524617 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:44:03.524583    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-4c29a2-openshift-default-c898b8b7b-2fgcd"]
Sep 17 14:44:05.493878 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:44:05.493842    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d882585-6a65-4e36-b1c7-7ec4562cd182" path="/var/lib/kubelet/pods/9d882585-6a65-4e36-b1c7-7ec4562cd182/volumes"
Sep 17 14:44:59.678049 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:44:59.678017    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-rr97k_45ffa13d-635a-46e8-8894-30902b9cf719/console-operator/2.log"
Sep 17 14:44:59.687992 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:44:59.687966    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-rr97k_45ffa13d-635a-46e8-8894-30902b9cf719/console-operator/2.log"
Sep 17 14:45:27.770134 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:45:27.770021    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:45:29.327270 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:45:29.327228    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:45:29.447872 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:45:29.447831    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6b6bd7d7cd-xsphd"]
Sep 17 14:45:29.448345 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:45:29.448325    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d7bc84f-6345-4a14-a73b-21e0aa0a30ff" containerName="istio-proxy"
Sep 17 14:45:29.448440 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:45:29.448347    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d7bc84f-6345-4a14-a73b-21e0aa0a30ff" containerName="istio-proxy"
Sep 17 14:45:29.448440 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:45:29.448361    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d882585-6a65-4e36-b1c7-7ec4562cd182" containerName="istio-proxy"
Sep 17 14:45:29.448440 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:45:29.448369    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d882585-6a65-4e36-b1c7-7ec4562cd182" containerName="istio-proxy"
Sep 17 14:45:29.448440 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:45:29.448387    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d7bc84f-6345-4a14-a73b-21e0aa0a30ff" containerName="istio-proxy"
Sep 17 14:45:29.448440 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:45:29.448397    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d7bc84f-6345-4a14-a73b-21e0aa0a30ff" containerName="istio-proxy"
Sep 17 14:45:29.448440 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:45:29.448411    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d078ec98-8e40-4f83-9fd8-6646f383eb16" containerName="revoke-keys"
Sep 17 14:45:29.448440 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:45:29.448419    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d078ec98-8e40-4f83-9fd8-6646f383eb16" containerName="revoke-keys"
Sep 17 14:45:29.448440 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:45:29.448431    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d882585-6a65-4e36-b1c7-7ec4562cd182" containerName="istio-proxy"
Sep 17 14:45:29.448440 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:45:29.448439    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d882585-6a65-4e36-b1c7-7ec4562cd182" containerName="istio-proxy"
Sep 17 14:45:29.448909 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:45:29.448541    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d078ec98-8e40-4f83-9fd8-6646f383eb16" containerName="revoke-keys"
Sep 17 14:45:29.448909 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:45:29.448554    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d882585-6a65-4e36-b1c7-7ec4562cd182" containerName="istio-proxy"
Sep 17 14:45:29.448909 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:45:29.448567    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d7bc84f-6345-4a14-a73b-21e0aa0a30ff" containerName="istio-proxy"
Sep 17 14:45:29.448909 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:45:29.448577    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d7bc84f-6345-4a14-a73b-21e0aa0a30ff" containerName="istio-proxy"
Sep 17 14:45:29.451944 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:45:29.451909    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b6bd7d7cd-xsphd"
Sep 17 14:45:29.454734 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:45:29.454711    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 17 14:45:29.454863 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:45:29.454829    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-p95mp\""
Sep 17 14:45:29.455563 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:45:29.455545    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 17 14:45:29.455712 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:45:29.455696    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 17 14:45:29.464143 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:45:29.464119    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b6bd7d7cd-xsphd"]
Sep 17 14:45:29.609567 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:45:29.609479    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bjc4\" (UniqueName: \"kubernetes.io/projected/4a187e6d-0790-47cd-b337-d19a069b06b6-kube-api-access-4bjc4\") pod \"maas-controller-6b6bd7d7cd-xsphd\" (UID: \"4a187e6d-0790-47cd-b337-d19a069b06b6\") " pod="opendatahub/maas-controller-6b6bd7d7cd-xsphd"
Sep 17 14:45:29.609567 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:45:29.609516    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a187e6d-0790-47cd-b337-d19a069b06b6-metrics-certs\") pod \"maas-controller-6b6bd7d7cd-xsphd\" (UID: \"4a187e6d-0790-47cd-b337-d19a069b06b6\") " pod="opendatahub/maas-controller-6b6bd7d7cd-xsphd"
Sep 17 14:45:29.609805 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:45:29.609641    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4a187e6d-0790-47cd-b337-d19a069b06b6-webhook-cert\") pod \"maas-controller-6b6bd7d7cd-xsphd\" (UID: \"4a187e6d-0790-47cd-b337-d19a069b06b6\") " pod="opendatahub/maas-controller-6b6bd7d7cd-xsphd"
Sep 17 14:45:29.710583 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:45:29.710548    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bjc4\" (UniqueName: \"kubernetes.io/projected/4a187e6d-0790-47cd-b337-d19a069b06b6-kube-api-access-4bjc4\") pod \"maas-controller-6b6bd7d7cd-xsphd\" (UID: \"4a187e6d-0790-47cd-b337-d19a069b06b6\") " pod="opendatahub/maas-controller-6b6bd7d7cd-xsphd"
Sep 17 14:45:29.710583 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:45:29.710584    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a187e6d-0790-47cd-b337-d19a069b06b6-metrics-certs\") pod \"maas-controller-6b6bd7d7cd-xsphd\" (UID: \"4a187e6d-0790-47cd-b337-d19a069b06b6\") " pod="opendatahub/maas-controller-6b6bd7d7cd-xsphd"
Sep 17 14:45:29.710817 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:45:29.710739    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4a187e6d-0790-47cd-b337-d19a069b06b6-webhook-cert\") pod \"maas-controller-6b6bd7d7cd-xsphd\" (UID: \"4a187e6d-0790-47cd-b337-d19a069b06b6\") " pod="opendatahub/maas-controller-6b6bd7d7cd-xsphd"
Sep 17 14:45:29.713190 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:45:29.713156    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4a187e6d-0790-47cd-b337-d19a069b06b6-webhook-cert\") pod \"maas-controller-6b6bd7d7cd-xsphd\" (UID: \"4a187e6d-0790-47cd-b337-d19a069b06b6\") " pod="opendatahub/maas-controller-6b6bd7d7cd-xsphd"
Sep 17 14:45:29.713319 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:45:29.713199    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a187e6d-0790-47cd-b337-d19a069b06b6-metrics-certs\") pod \"maas-controller-6b6bd7d7cd-xsphd\" (UID: \"4a187e6d-0790-47cd-b337-d19a069b06b6\") " pod="opendatahub/maas-controller-6b6bd7d7cd-xsphd"
Sep 17 14:45:29.720547 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:45:29.720522    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bjc4\" (UniqueName: \"kubernetes.io/projected/4a187e6d-0790-47cd-b337-d19a069b06b6-kube-api-access-4bjc4\") pod \"maas-controller-6b6bd7d7cd-xsphd\" (UID: \"4a187e6d-0790-47cd-b337-d19a069b06b6\") " pod="opendatahub/maas-controller-6b6bd7d7cd-xsphd"
Sep 17 14:45:29.763360 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:45:29.763323    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b6bd7d7cd-xsphd"
Sep 17 14:45:30.040421 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:45:30.040392    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b6bd7d7cd-xsphd"]
Sep 17 14:45:30.042664 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:45:30.042638    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a187e6d_0790_47cd_b337_d19a069b06b6.slice/crio-100e97a71e448671d28bbf9f25e55ebb2072497b90b93fc3fcd8616812ad9a0f WatchSource:0}: Error finding container 100e97a71e448671d28bbf9f25e55ebb2072497b90b93fc3fcd8616812ad9a0f: Status 404 returned error can't find the container with id 100e97a71e448671d28bbf9f25e55ebb2072497b90b93fc3fcd8616812ad9a0f
Sep 17 14:45:30.044080 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:45:30.044063    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 14:45:30.976155 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:45:30.976115    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b6bd7d7cd-xsphd" event={"ID":"4a187e6d-0790-47cd-b337-d19a069b06b6","Type":"ContainerStarted","Data":"100e97a71e448671d28bbf9f25e55ebb2072497b90b93fc3fcd8616812ad9a0f"}
Sep 17 14:45:32.990424 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:45:32.990385    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b6bd7d7cd-xsphd" event={"ID":"4a187e6d-0790-47cd-b337-d19a069b06b6","Type":"ContainerStarted","Data":"758fb44f5a5b698c391804aa413f7394ce1bc141d35f11875f9a214b4ebbf828"}
Sep 17 14:45:32.990859 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:45:32.990530    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6b6bd7d7cd-xsphd"
Sep 17 14:45:33.014266 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:45:33.014210    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6b6bd7d7cd-xsphd" podStartSLOduration=1.687309427 podStartE2EDuration="4.014195958s" totalImagesPullingTime="2.326886531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:45:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:45:33.012243913 +0000 UTC m=+2434.013787388" watchObservedRunningTime="2026-09-17 14:45:33.014195958 +0000 UTC m=+2434.015739433"
Sep 17 14:45:43.998896 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:45:43.998861    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6b6bd7d7cd-xsphd"
Sep 17 14:46:58.358274 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:46:58.358186    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6lzpm"]
Sep 17 14:46:58.359779 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:46:58.358424    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6lzpm" podUID="3a99f184-10e9-401d-b502-4bc6889b8d2b" containerName="manager" containerID="cri-o://cf3d4e8fd20829b33b290d1ea0b65bd462b4a52cac0c93f56c485d9caa3181fc" gracePeriod=10
Sep 17 14:46:58.619604 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:46:58.619543    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6lzpm"
Sep 17 14:46:58.748564 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:46:58.748532    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3a99f184-10e9-401d-b502-4bc6889b8d2b-extensions-socket-volume\") pod \"3a99f184-10e9-401d-b502-4bc6889b8d2b\" (UID: \"3a99f184-10e9-401d-b502-4bc6889b8d2b\") "
Sep 17 14:46:58.748767 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:46:58.748576    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nkszd\" (UniqueName: \"kubernetes.io/projected/3a99f184-10e9-401d-b502-4bc6889b8d2b-kube-api-access-nkszd\") pod \"3a99f184-10e9-401d-b502-4bc6889b8d2b\" (UID: \"3a99f184-10e9-401d-b502-4bc6889b8d2b\") "
Sep 17 14:46:58.749007 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:46:58.748976    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a99f184-10e9-401d-b502-4bc6889b8d2b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3a99f184-10e9-401d-b502-4bc6889b8d2b" (UID: "3a99f184-10e9-401d-b502-4bc6889b8d2b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:46:58.751308 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:46:58.751272    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a99f184-10e9-401d-b502-4bc6889b8d2b-kube-api-access-nkszd" (OuterVolumeSpecName: "kube-api-access-nkszd") pod "3a99f184-10e9-401d-b502-4bc6889b8d2b" (UID: "3a99f184-10e9-401d-b502-4bc6889b8d2b"). InnerVolumeSpecName "kube-api-access-nkszd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:46:58.849753 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:46:58.849706    2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3a99f184-10e9-401d-b502-4bc6889b8d2b-extensions-socket-volume\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:46:58.849753 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:46:58.849750    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkszd\" (UniqueName: \"kubernetes.io/projected/3a99f184-10e9-401d-b502-4bc6889b8d2b-kube-api-access-nkszd\") on node \"ip-10-0-137-53.ec2.internal\" DevicePath \"\""
Sep 17 14:46:59.475376 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:46:59.475334    2582 generic.go:356] "Generic (PLEG): container finished" podID="3a99f184-10e9-401d-b502-4bc6889b8d2b" containerID="cf3d4e8fd20829b33b290d1ea0b65bd462b4a52cac0c93f56c485d9caa3181fc" exitCode=0
Sep 17 14:46:59.475376 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:46:59.475383    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6lzpm"
Sep 17 14:46:59.475876 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:46:59.475426    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6lzpm" event={"ID":"3a99f184-10e9-401d-b502-4bc6889b8d2b","Type":"ContainerDied","Data":"cf3d4e8fd20829b33b290d1ea0b65bd462b4a52cac0c93f56c485d9caa3181fc"}
Sep 17 14:46:59.475876 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:46:59.475487    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6lzpm" event={"ID":"3a99f184-10e9-401d-b502-4bc6889b8d2b","Type":"ContainerDied","Data":"b44258ae9b8d147806003ea892e03f7291fefffe6e9c3957f243b48233407950"}
Sep 17 14:46:59.475876 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:46:59.475509    2582 scope.go:117] "RemoveContainer" containerID="cf3d4e8fd20829b33b290d1ea0b65bd462b4a52cac0c93f56c485d9caa3181fc"
Sep 17 14:46:59.485185 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:46:59.485164    2582 scope.go:117] "RemoveContainer" containerID="cf3d4e8fd20829b33b290d1ea0b65bd462b4a52cac0c93f56c485d9caa3181fc"
Sep 17 14:46:59.485484 ip-10-0-137-53 kubenswrapper[2582]: E0917 14:46:59.485462    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf3d4e8fd20829b33b290d1ea0b65bd462b4a52cac0c93f56c485d9caa3181fc\": container with ID starting with cf3d4e8fd20829b33b290d1ea0b65bd462b4a52cac0c93f56c485d9caa3181fc not found: ID does not exist" containerID="cf3d4e8fd20829b33b290d1ea0b65bd462b4a52cac0c93f56c485d9caa3181fc"
Sep 17 14:46:59.485548 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:46:59.485492    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf3d4e8fd20829b33b290d1ea0b65bd462b4a52cac0c93f56c485d9caa3181fc"} err="failed to get container status \"cf3d4e8fd20829b33b290d1ea0b65bd462b4a52cac0c93f56c485d9caa3181fc\": rpc error: code = NotFound desc = could not find container \"cf3d4e8fd20829b33b290d1ea0b65bd462b4a52cac0c93f56c485d9caa3181fc\": container with ID starting with cf3d4e8fd20829b33b290d1ea0b65bd462b4a52cac0c93f56c485d9caa3181fc not found: ID does not exist"
Sep 17 14:46:59.500466 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:46:59.500437    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6lzpm"]
Sep 17 14:46:59.503673 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:46:59.503639    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6lzpm"]
Sep 17 14:47:01.494261 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:47:01.494229    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a99f184-10e9-401d-b502-4bc6889b8d2b" path="/var/lib/kubelet/pods/3a99f184-10e9-401d-b502-4bc6889b8d2b/volumes"
Sep 17 14:47:59.062219 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:47:59.062174    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hz254"]
Sep 17 14:47:59.062845 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:47:59.062795    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a99f184-10e9-401d-b502-4bc6889b8d2b" containerName="manager"
Sep 17 14:47:59.062845 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:47:59.062817    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a99f184-10e9-401d-b502-4bc6889b8d2b" containerName="manager"
Sep 17 14:47:59.063007 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:47:59.062993    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a99f184-10e9-401d-b502-4bc6889b8d2b" containerName="manager"
Sep 17 14:47:59.063072 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:47:59.063013    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d882585-6a65-4e36-b1c7-7ec4562cd182" containerName="istio-proxy"
Sep 17 14:47:59.065160 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:47:59.065139    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hz254"
Sep 17 14:47:59.067797 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:47:59.067779    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-n6xsv\""
Sep 17 14:47:59.077209 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:47:59.077188    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hz254"]
Sep 17 14:47:59.086808 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:47:59.086781    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b0956a2f-72f3-43f7-9223-25be4260661e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-hz254\" (UID: \"b0956a2f-72f3-43f7-9223-25be4260661e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hz254"
Sep 17 14:47:59.086945 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:47:59.086927    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mrs6\" (UniqueName: \"kubernetes.io/projected/b0956a2f-72f3-43f7-9223-25be4260661e-kube-api-access-6mrs6\") pod \"kuadrant-operator-controller-manager-5769f99d9f-hz254\" (UID: \"b0956a2f-72f3-43f7-9223-25be4260661e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hz254"
Sep 17 14:47:59.187527 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:47:59.187492    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b0956a2f-72f3-43f7-9223-25be4260661e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-hz254\" (UID: \"b0956a2f-72f3-43f7-9223-25be4260661e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hz254"
Sep 17 14:47:59.187723 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:47:59.187584    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mrs6\" (UniqueName: \"kubernetes.io/projected/b0956a2f-72f3-43f7-9223-25be4260661e-kube-api-access-6mrs6\") pod \"kuadrant-operator-controller-manager-5769f99d9f-hz254\" (UID: \"b0956a2f-72f3-43f7-9223-25be4260661e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hz254"
Sep 17 14:47:59.187910 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:47:59.187888    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b0956a2f-72f3-43f7-9223-25be4260661e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-hz254\" (UID: \"b0956a2f-72f3-43f7-9223-25be4260661e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hz254"
Sep 17 14:47:59.196258 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:47:59.196231    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mrs6\" (UniqueName: \"kubernetes.io/projected/b0956a2f-72f3-43f7-9223-25be4260661e-kube-api-access-6mrs6\") pod \"kuadrant-operator-controller-manager-5769f99d9f-hz254\" (UID: \"b0956a2f-72f3-43f7-9223-25be4260661e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hz254"
Sep 17 14:47:59.378927 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:47:59.378850    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-n6xsv\""
Sep 17 14:47:59.387242 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:47:59.387225    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hz254"
Sep 17 14:47:59.667155 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:47:59.666967    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hz254"]
Sep 17 14:47:59.670521 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:47:59.670492    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0956a2f_72f3_43f7_9223_25be4260661e.slice/crio-8dba71c49f9d97ceeb4772eae32f390d248b5cd294a49904c70321ecf97ecc39 WatchSource:0}: Error finding container 8dba71c49f9d97ceeb4772eae32f390d248b5cd294a49904c70321ecf97ecc39: Status 404 returned error can't find the container with id 8dba71c49f9d97ceeb4772eae32f390d248b5cd294a49904c70321ecf97ecc39
Sep 17 14:47:59.797616 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:47:59.797578    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hz254" event={"ID":"b0956a2f-72f3-43f7-9223-25be4260661e","Type":"ContainerStarted","Data":"11cc87b622d2f054c30d10c235ac5a76c21a8a0b59de86ec07f51841dd5ee0a2"}
Sep 17 14:47:59.797749 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:47:59.797648    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hz254" event={"ID":"b0956a2f-72f3-43f7-9223-25be4260661e","Type":"ContainerStarted","Data":"8dba71c49f9d97ceeb4772eae32f390d248b5cd294a49904c70321ecf97ecc39"}
Sep 17 14:47:59.797799 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:47:59.797758    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hz254"
Sep 17 14:47:59.829416 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:47:59.829358    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hz254" podStartSLOduration=0.829341147 podStartE2EDuration="829.341147ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:47:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:47:59.827542216 +0000 UTC m=+2580.829085718" watchObservedRunningTime="2026-09-17 14:47:59.829341147 +0000 UTC m=+2580.830884620"
Sep 17 14:48:10.284811 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:48:10.284770    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:48:10.805473 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:48:10.805439    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hz254"
Sep 17 14:48:11.759366 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:48:11.759315    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:48:52.844548 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:48:52.844446    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:49:00.186812 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:49:00.186775    2582 scope.go:117] "RemoveContainer" containerID="21886ed6b21e90a492e098b91537e197a93f381ca75235b752e5bb31ad0b2393"
Sep 17 14:49:01.779244 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:49:01.779199    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:49:12.431418 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:49:12.431362    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:49:34.415156 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:49:34.415115    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:49:45.092920 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:49:45.092884    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:49:46.060429 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:49:46.060392    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:49:59.712184 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:49:59.712103    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-rr97k_45ffa13d-635a-46e8-8894-30902b9cf719/console-operator/2.log"
Sep 17 14:49:59.722161 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:49:59.722132    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-rr97k_45ffa13d-635a-46e8-8894-30902b9cf719/console-operator/2.log"
Sep 17 14:50:00.208120 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:50:00.208091    2582 scope.go:117] "RemoveContainer" containerID="4b57f8d1184f880815f0c3a257d0ca834076f55ae1823068e858eff8fed1651a"
Sep 17 14:50:37.096944 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:50:37.096913    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-64c7cbf655-rsx9d_f403d577-cab9-41f5-b000-f8ae11ffabee/authorino/0.log"
Sep 17 14:50:42.541668 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:50:42.541597    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-l5q8z_328fc0b9-c0f0-4e49-9756-2b88cb1f9576/manager/0.log"
Sep 17 14:50:42.668785 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:50:42.668752    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6b6bd7d7cd-xsphd_4a187e6d-0790-47cd-b337-d19a069b06b6/manager/0.log"
Sep 17 14:50:42.798481 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:50:42.798401    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-hpmc6_839e8ecf-313f-4dac-965a-9c071d3af679/server/0.log"
Sep 17 14:50:42.919155 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:50:42.919125    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-mdqnr_d61faf19-d78e-494d-928c-27a2ffbe33c0/manager/0.log"
Sep 17 14:50:43.039646 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:50:43.039596    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7895599765-fb8q4_caf36299-408c-4b4c-a5f3-f6125324fc7e/copy-manifests/0.log"
Sep 17 14:50:43.046430 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:50:43.046405    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7895599765-fb8q4_caf36299-408c-4b4c-a5f3-f6125324fc7e/manager/0.log"
Sep 17 14:50:43.926932 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:50:43.926900    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29827590-ncd6j_82b52f92-7f30-4db4-b1fc-b013003c0e13/cleanup/0.log"
Sep 17 14:50:44.187171 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:50:44.187092    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-67kml_8aaf43e4-c249-416a-b673-d16a0dc79e7d/postgres/0.log"
Sep 17 14:50:44.969553 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:50:44.969521    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-64c7cbf655-rsx9d_f403d577-cab9-41f5-b000-f8ae11ffabee/authorino/0.log"
Sep 17 14:50:45.212443 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:50:45.212411    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-v7j87_0a804f40-44d8-4775-8aa6-43cef6da55aa/manager/0.log"
Sep 17 14:50:45.443119 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:50:45.443052    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-hz254_b0956a2f-72f3-43f7-9223-25be4260661e/manager/0.log"
Sep 17 14:50:46.064388 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:50:46.064359    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn_d76458e2-4b38-4f8b-947a-839acc027b9e/istio-proxy/0.log"
Sep 17 14:50:46.536375 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:50:46.536347    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-znmmb_612ce2f4-8e66-49cd-ad34-acab04c857d2/istio-proxy/1.log"
Sep 17 14:50:46.660036 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:50:46.660006    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-9x9j4_8afff3a4-0475-4e19-96a8-562397330290/payload-pre-processing/0.log"
Sep 17 14:50:46.909732 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:50:46.909656    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-68854cb984-hm4cb_1414b9c2-7c96-4383-9947-e7abcd68a074/router/0.log"
Sep 17 14:50:53.166804 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:50:53.166767    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-82mm4/must-gather-x8kz6"]
Sep 17 14:50:53.169407 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:50:53.169385    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-82mm4/must-gather-x8kz6"
Sep 17 14:50:53.171955 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:50:53.171933    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-82mm4\"/\"kube-root-ca.crt\""
Sep 17 14:50:53.172092 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:50:53.171934    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-82mm4\"/\"openshift-service-ca.crt\""
Sep 17 14:50:53.172175 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:50:53.172158    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-82mm4\"/\"default-dockercfg-qm296\""
Sep 17 14:50:53.179566 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:50:53.179542    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-82mm4/must-gather-x8kz6"]
Sep 17 14:50:53.191374 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:50:53.191345    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:50:53.211888 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:50:53.211853    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/8105f67a-9c2c-4223-bee5-d5de554d949b-must-gather-output\") pod \"must-gather-x8kz6\" (UID: \"8105f67a-9c2c-4223-bee5-d5de554d949b\") " pod="openshift-must-gather-82mm4/must-gather-x8kz6"
Sep 17 14:50:53.212035 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:50:53.211914    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26wtg\" (UniqueName: \"kubernetes.io/projected/8105f67a-9c2c-4223-bee5-d5de554d949b-kube-api-access-26wtg\") pod \"must-gather-x8kz6\" (UID: \"8105f67a-9c2c-4223-bee5-d5de554d949b\") " pod="openshift-must-gather-82mm4/must-gather-x8kz6"
Sep 17 14:50:53.286390 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:50:53.286348    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log" limit=1024
Sep 17 14:50:53.312486 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:50:53.312444    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/8105f67a-9c2c-4223-bee5-d5de554d949b-must-gather-output\") pod \"must-gather-x8kz6\" (UID: \"8105f67a-9c2c-4223-bee5-d5de554d949b\") " pod="openshift-must-gather-82mm4/must-gather-x8kz6"
Sep 17 14:50:53.312710 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:50:53.312509    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26wtg\" (UniqueName: \"kubernetes.io/projected/8105f67a-9c2c-4223-bee5-d5de554d949b-kube-api-access-26wtg\") pod \"must-gather-x8kz6\" (UID: \"8105f67a-9c2c-4223-bee5-d5de554d949b\") " pod="openshift-must-gather-82mm4/must-gather-x8kz6"
Sep 17 14:50:53.312904 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:50:53.312881    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/8105f67a-9c2c-4223-bee5-d5de554d949b-must-gather-output\") pod \"must-gather-x8kz6\" (UID: \"8105f67a-9c2c-4223-bee5-d5de554d949b\") " pod="openshift-must-gather-82mm4/must-gather-x8kz6"
Sep 17 14:50:53.325888 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:50:53.325868    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26wtg\" (UniqueName: \"kubernetes.io/projected/8105f67a-9c2c-4223-bee5-d5de554d949b-kube-api-access-26wtg\") pod \"must-gather-x8kz6\" (UID: \"8105f67a-9c2c-4223-bee5-d5de554d949b\") " pod="openshift-must-gather-82mm4/must-gather-x8kz6"
Sep 17 14:50:53.478952 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:50:53.478919    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-82mm4/must-gather-x8kz6"
Sep 17 14:50:53.751071 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:50:53.751041    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-82mm4/must-gather-x8kz6"]
Sep 17 14:50:53.753635 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:50:53.753588    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8105f67a_9c2c_4223_bee5_d5de554d949b.slice/crio-265237a8b67a22b469dcc2a6d05a48dcab6f6c2654c47a1617f2069c7619980c WatchSource:0}: Error finding container 265237a8b67a22b469dcc2a6d05a48dcab6f6c2654c47a1617f2069c7619980c: Status 404 returned error can't find the container with id 265237a8b67a22b469dcc2a6d05a48dcab6f6c2654c47a1617f2069c7619980c
Sep 17 14:50:53.755488 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:50:53.755472    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 14:50:54.757234 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:50:54.757051    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-82mm4/must-gather-x8kz6" event={"ID":"8105f67a-9c2c-4223-bee5-d5de554d949b","Type":"ContainerStarted","Data":"917b4c301881e863610bc28260120a84dad6c579f70475e28eee144e13c319e9"}
Sep 17 14:50:54.757234 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:50:54.757107    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-82mm4/must-gather-x8kz6" event={"ID":"8105f67a-9c2c-4223-bee5-d5de554d949b","Type":"ContainerStarted","Data":"6d7dcbd6fe07925ecb1a371daac98fe08994a2149a25f6f43791c17ebb6cfcc1"}
Sep 17 14:50:54.757234 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:50:54.757124    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-82mm4/must-gather-x8kz6" event={"ID":"8105f67a-9c2c-4223-bee5-d5de554d949b","Type":"ContainerStarted","Data":"265237a8b67a22b469dcc2a6d05a48dcab6f6c2654c47a1617f2069c7619980c"}
Sep 17 14:50:54.778370 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:50:54.778257    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-82mm4/must-gather-x8kz6" podStartSLOduration=1.012787857 podStartE2EDuration="1.778232929s" totalImagesPullingTime="765.445072ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:50:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:50:54.774165316 +0000 UTC m=+2755.775708782" watchObservedRunningTime="2026-09-17 14:50:54.778232929 +0000 UTC m=+2755.779776502"
Sep 17 14:50:56.173454 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:50:56.173422    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-7k2wk_31e72879-bbfc-4684-bcbb-fd79b4eec4b6/global-pull-secret-syncer/0.log"
Sep 17 14:50:56.293486 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:50:56.293452    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-2j77v_bc013ec0-cbfe-4335-a5c1-a32d9a3f0a5f/konnectivity-agent/0.log"
Sep 17 14:50:56.460001 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:50:56.459966    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-53.ec2.internal_3ed51660b0e0cea0ccab4eff146b4e99/haproxy/0.log"
Sep 17 14:51:01.091243 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:01.090708    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-64c7cbf655-rsx9d_f403d577-cab9-41f5-b000-f8ae11ffabee/authorino/0.log"
Sep 17 14:51:01.145646 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:01.143829    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-v7j87_0a804f40-44d8-4775-8aa6-43cef6da55aa/manager/0.log"
Sep 17 14:51:01.217546 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:01.217508    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-hz254_b0956a2f-72f3-43f7-9223-25be4260661e/manager/0.log"
Sep 17 14:51:02.889912 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:02.889876    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cd954b897-r58pd_1c6950fd-ebfd-4e8e-acbe-6c6e72c00b99/cluster-monitoring-operator/0.log"
Sep 17 14:51:02.917527 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:02.917484    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-5qnzn_c187d7ca-b3c3-4fe0-9e5c-a2c9fc28c347/kube-state-metrics/0.log"
Sep 17 14:51:02.935751 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:02.935723    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-5qnzn_c187d7ca-b3c3-4fe0-9e5c-a2c9fc28c347/kube-rbac-proxy-main/0.log"
Sep 17 14:51:02.959582 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:02.959536    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-5qnzn_c187d7ca-b3c3-4fe0-9e5c-a2c9fc28c347/kube-rbac-proxy-self/0.log"
Sep 17 14:51:02.990794 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:02.990764    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-67d9d56b4b-545xf_840db932-27b6-4ac5-8420-926b7277f4a3/metrics-server/0.log"
Sep 17 14:51:03.022800 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:03.022769    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5d85778b9d-tcmcl_214c3411-12f9-49c7-9bcd-01eca91e612b/monitoring-plugin/0.log"
Sep 17 14:51:03.261401 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:03.261353    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vh2mf_357b3c15-b702-46dd-949f-d9c2fcd8872d/node-exporter/0.log"
Sep 17 14:51:03.299904 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:03.297186    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vh2mf_357b3c15-b702-46dd-949f-d9c2fcd8872d/kube-rbac-proxy/0.log"
Sep 17 14:51:03.350120 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:03.350044    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vh2mf_357b3c15-b702-46dd-949f-d9c2fcd8872d/init-textfile/0.log"
Sep 17 14:51:03.387448 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:03.387422    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-6hsrq_fc1de48c-04a5-4a21-be57-66a4e91b3947/kube-rbac-proxy-main/0.log"
Sep 17 14:51:03.415657 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:03.415586    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-6hsrq_fc1de48c-04a5-4a21-be57-66a4e91b3947/kube-rbac-proxy-self/0.log"
Sep 17 14:51:03.458106 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:03.458017    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-6hsrq_fc1de48c-04a5-4a21-be57-66a4e91b3947/openshift-state-metrics/0.log"
Sep 17 14:51:03.706674 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:03.706636    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5864cd99d7-9ppw7_8574e419-b1dd-4888-a942-f7c38df4795d/prometheus-operator-admission-webhook/0.log"
Sep 17 14:51:03.736921 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:03.736830    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7cc4cdf7fc-ngzrk_1c33a78b-d753-426f-bbd0-10acad2ee10b/telemeter-client/0.log"
Sep 17 14:51:03.756599 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:03.756561    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7cc4cdf7fc-ngzrk_1c33a78b-d753-426f-bbd0-10acad2ee10b/reload/0.log"
Sep 17 14:51:03.777326 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:03.777284    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7cc4cdf7fc-ngzrk_1c33a78b-d753-426f-bbd0-10acad2ee10b/kube-rbac-proxy/0.log"
Sep 17 14:51:05.355226 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:05.355189    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-5dcbc7df6b-7hhfj_b9b37579-a215-4a77-bd0a-f33675b1b17f/networking-console-plugin/0.log"
Sep 17 14:51:05.964912 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:05.964876    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-rr97k_45ffa13d-635a-46e8-8894-30902b9cf719/console-operator/2.log"
Sep 17 14:51:05.975530 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:05.975497    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-rr97k_45ffa13d-635a-46e8-8894-30902b9cf719/console-operator/3.log"
Sep 17 14:51:06.433796 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:06.433693    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-82mm4/perf-node-gather-daemonset-hh6hq"]
Sep 17 14:51:06.440745 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:06.440711    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-82mm4/perf-node-gather-daemonset-hh6hq"
Sep 17 14:51:06.447103 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:06.447063    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-82mm4/perf-node-gather-daemonset-hh6hq"]
Sep 17 14:51:06.589295 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:06.589259    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fc31d121-c3b0-4936-aea5-28aa97703b5f-proc\") pod \"perf-node-gather-daemonset-hh6hq\" (UID: \"fc31d121-c3b0-4936-aea5-28aa97703b5f\") " pod="openshift-must-gather-82mm4/perf-node-gather-daemonset-hh6hq"
Sep 17 14:51:06.589858 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:06.589835    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4lld\" (UniqueName: \"kubernetes.io/projected/fc31d121-c3b0-4936-aea5-28aa97703b5f-kube-api-access-v4lld\") pod \"perf-node-gather-daemonset-hh6hq\" (UID: \"fc31d121-c3b0-4936-aea5-28aa97703b5f\") " pod="openshift-must-gather-82mm4/perf-node-gather-daemonset-hh6hq"
Sep 17 14:51:06.590128 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:06.590113    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fc31d121-c3b0-4936-aea5-28aa97703b5f-podres\") pod \"perf-node-gather-daemonset-hh6hq\" (UID: \"fc31d121-c3b0-4936-aea5-28aa97703b5f\") " pod="openshift-must-gather-82mm4/perf-node-gather-daemonset-hh6hq"
Sep 17 14:51:06.590280 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:06.590264    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fc31d121-c3b0-4936-aea5-28aa97703b5f-sys\") pod \"perf-node-gather-daemonset-hh6hq\" (UID: \"fc31d121-c3b0-4936-aea5-28aa97703b5f\") " pod="openshift-must-gather-82mm4/perf-node-gather-daemonset-hh6hq"
Sep 17 14:51:06.590472 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:06.590458    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fc31d121-c3b0-4936-aea5-28aa97703b5f-lib-modules\") pod \"perf-node-gather-daemonset-hh6hq\" (UID: \"fc31d121-c3b0-4936-aea5-28aa97703b5f\") " pod="openshift-must-gather-82mm4/perf-node-gather-daemonset-hh6hq"
Sep 17 14:51:06.593058 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:06.593027    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-567467b958-q9h5x_3c4bd392-74a7-41ce-acf8-de0ff81c4a44/console/0.log"
Sep 17 14:51:06.628693 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:06.628660    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-76f99584f4-82kkv_c061e677-4341-4e7f-baa8-5d41b143d332/download-server/0.log"
Sep 17 14:51:06.693114 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:06.691977    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fc31d121-c3b0-4936-aea5-28aa97703b5f-lib-modules\") pod \"perf-node-gather-daemonset-hh6hq\" (UID: \"fc31d121-c3b0-4936-aea5-28aa97703b5f\") " pod="openshift-must-gather-82mm4/perf-node-gather-daemonset-hh6hq"
Sep 17 14:51:06.693114 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:06.692062    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fc31d121-c3b0-4936-aea5-28aa97703b5f-proc\") pod \"perf-node-gather-daemonset-hh6hq\" (UID: \"fc31d121-c3b0-4936-aea5-28aa97703b5f\") " pod="openshift-must-gather-82mm4/perf-node-gather-daemonset-hh6hq"
Sep 17 14:51:06.693114 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:06.692101    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4lld\" (UniqueName: \"kubernetes.io/projected/fc31d121-c3b0-4936-aea5-28aa97703b5f-kube-api-access-v4lld\") pod \"perf-node-gather-daemonset-hh6hq\" (UID: \"fc31d121-c3b0-4936-aea5-28aa97703b5f\") " pod="openshift-must-gather-82mm4/perf-node-gather-daemonset-hh6hq"
Sep 17 14:51:06.693114 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:06.692169    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fc31d121-c3b0-4936-aea5-28aa97703b5f-podres\") pod \"perf-node-gather-daemonset-hh6hq\" (UID: \"fc31d121-c3b0-4936-aea5-28aa97703b5f\") " pod="openshift-must-gather-82mm4/perf-node-gather-daemonset-hh6hq"
Sep 17 14:51:06.693114 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:06.692200    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fc31d121-c3b0-4936-aea5-28aa97703b5f-sys\") pod \"perf-node-gather-daemonset-hh6hq\" (UID: \"fc31d121-c3b0-4936-aea5-28aa97703b5f\") " pod="openshift-must-gather-82mm4/perf-node-gather-daemonset-hh6hq"
Sep 17 14:51:06.693114 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:06.692329    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fc31d121-c3b0-4936-aea5-28aa97703b5f-sys\") pod \"perf-node-gather-daemonset-hh6hq\" (UID: \"fc31d121-c3b0-4936-aea5-28aa97703b5f\") " pod="openshift-must-gather-82mm4/perf-node-gather-daemonset-hh6hq"
Sep 17 14:51:06.693114 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:06.692470    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fc31d121-c3b0-4936-aea5-28aa97703b5f-lib-modules\") pod \"perf-node-gather-daemonset-hh6hq\" (UID: \"fc31d121-c3b0-4936-aea5-28aa97703b5f\") " pod="openshift-must-gather-82mm4/perf-node-gather-daemonset-hh6hq"
Sep 17 14:51:06.693114 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:06.692530    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fc31d121-c3b0-4936-aea5-28aa97703b5f-proc\") pod \"perf-node-gather-daemonset-hh6hq\" (UID: \"fc31d121-c3b0-4936-aea5-28aa97703b5f\") " pod="openshift-must-gather-82mm4/perf-node-gather-daemonset-hh6hq"
Sep 17 14:51:06.693114 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:06.693005    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fc31d121-c3b0-4936-aea5-28aa97703b5f-podres\") pod \"perf-node-gather-daemonset-hh6hq\" (UID: \"fc31d121-c3b0-4936-aea5-28aa97703b5f\") " pod="openshift-must-gather-82mm4/perf-node-gather-daemonset-hh6hq"
Sep 17 14:51:06.703083 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:06.703014    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4lld\" (UniqueName: \"kubernetes.io/projected/fc31d121-c3b0-4936-aea5-28aa97703b5f-kube-api-access-v4lld\") pod \"perf-node-gather-daemonset-hh6hq\" (UID: \"fc31d121-c3b0-4936-aea5-28aa97703b5f\") " pod="openshift-must-gather-82mm4/perf-node-gather-daemonset-hh6hq"
Sep 17 14:51:06.755592 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:06.755549    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-82mm4/perf-node-gather-daemonset-hh6hq"
Sep 17 14:51:07.055850 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:07.055682    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-82mm4/perf-node-gather-daemonset-hh6hq"]
Sep 17 14:51:07.058080 ip-10-0-137-53 kubenswrapper[2582]: W0917 14:51:07.058045    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podfc31d121_c3b0_4936_aea5_28aa97703b5f.slice/crio-884b97e1bc98a33b3d705ad75a59b7b280c7b7ab08574e969301886aabb9a880 WatchSource:0}: Error finding container 884b97e1bc98a33b3d705ad75a59b7b280c7b7ab08574e969301886aabb9a880: Status 404 returned error can't find the container with id 884b97e1bc98a33b3d705ad75a59b7b280c7b7ab08574e969301886aabb9a880
Sep 17 14:51:07.332484 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:07.332391    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-86695b7cc7-dhfb8_83ecb87d-85fa-4ac1-9800-103030f2d959/volume-data-source-validator/0.log"
Sep 17 14:51:07.868500 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:07.868456    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-82mm4/perf-node-gather-daemonset-hh6hq" event={"ID":"fc31d121-c3b0-4936-aea5-28aa97703b5f","Type":"ContainerStarted","Data":"4b9672b8d8914a4c18dd5ab9c0f935f781ea164146e70f0aadd365e6ff80cc10"}
Sep 17 14:51:07.868989 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:07.868525    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-82mm4/perf-node-gather-daemonset-hh6hq" event={"ID":"fc31d121-c3b0-4936-aea5-28aa97703b5f","Type":"ContainerStarted","Data":"884b97e1bc98a33b3d705ad75a59b7b280c7b7ab08574e969301886aabb9a880"}
Sep 17 14:51:07.868989 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:07.868567    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-82mm4/perf-node-gather-daemonset-hh6hq"
Sep 17 14:51:07.889896 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:07.889834    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-82mm4/perf-node-gather-daemonset-hh6hq" podStartSLOduration=1.889812657 podStartE2EDuration="1.889812657s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 14:51:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:51:07.886976099 +0000 UTC m=+2768.888519575" watchObservedRunningTime="2026-09-17 14:51:07.889812657 +0000 UTC m=+2768.891356133"
Sep 17 14:51:08.495274 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:08.495231    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6v652_40b77f59-aa77-4958-bd61-2522680f1ecf/dns/0.log"
Sep 17 14:51:08.518160 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:08.518128    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6v652_40b77f59-aa77-4958-bd61-2522680f1ecf/kube-rbac-proxy/0.log"
Sep 17 14:51:08.694109 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:08.694076    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-s2nlk_1268d6c2-0f80-41e0-98d5-d72d5c03da1b/dns-node-resolver/0.log"
Sep 17 14:51:09.323939 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:09.323901    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-72skg_4462f6aa-ccc5-42d0-8623-f0dad6105560/node-ca/0.log"
Sep 17 14:51:10.296453 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:10.296425    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn_d76458e2-4b38-4f8b-947a-839acc027b9e/istio-proxy/0.log"
Sep 17 14:51:11.101174 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:11.101131    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-znmmb_612ce2f4-8e66-49cd-ad34-acab04c857d2/istio-proxy/0.log"
Sep 17 14:51:11.105714 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:11.105687    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-znmmb_612ce2f4-8e66-49cd-ad34-acab04c857d2/istio-proxy/1.log"
Sep 17 14:51:11.146935 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:11.146893    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-9x9j4_8afff3a4-0475-4e19-96a8-562397330290/payload-pre-processing/0.log"
Sep 17 14:51:11.220098 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:11.220071    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-68854cb984-hm4cb_1414b9c2-7c96-4383-9947-e7abcd68a074/router/0.log"
Sep 17 14:51:12.112407 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:12.112374    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zjz6h_5cd5a40b-4883-4928-8dd6-b0c3d13ad8ed/serve-healthcheck-canary/0.log"
Sep 17 14:51:12.621229 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:12.621192    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-thrb2_903d323b-ec37-4e01-89e9-7ab8caa061d7/insights-operator/0.log"
Sep 17 14:51:12.621717 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:12.621696    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-thrb2_903d323b-ec37-4e01-89e9-7ab8caa061d7/insights-operator/1.log"
Sep 17 14:51:12.643816 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:12.643782    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4dt5r_5cd9349e-5c41-4588-98e3-35fbe230801c/kube-rbac-proxy/0.log"
Sep 17 14:51:12.662055 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:12.662027    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4dt5r_5cd9349e-5c41-4588-98e3-35fbe230801c/exporter/0.log"
Sep 17 14:51:12.682791 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:12.682757    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4dt5r_5cd9349e-5c41-4588-98e3-35fbe230801c/extractor/0.log"
Sep 17 14:51:13.887991 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:13.887962    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-82mm4/perf-node-gather-daemonset-hh6hq"
Sep 17 14:51:15.256315 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:15.256216    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-l5q8z_328fc0b9-c0f0-4e49-9756-2b88cb1f9576/manager/0.log"
Sep 17 14:51:15.316105 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:15.316011    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6b6bd7d7cd-xsphd_4a187e6d-0790-47cd-b337-d19a069b06b6/manager/0.log"
Sep 17 14:51:15.337200 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:15.337162    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-hpmc6_839e8ecf-313f-4dac-965a-9c071d3af679/server/0.log"
Sep 17 14:51:15.441394 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:15.441352    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-mdqnr_d61faf19-d78e-494d-928c-27a2ffbe33c0/manager/0.log"
Sep 17 14:51:15.476633 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:15.474279    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7895599765-fb8q4_caf36299-408c-4b4c-a5f3-f6125324fc7e/manager/0.log"
Sep 17 14:51:15.494091 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:15.494053    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7895599765-fb8q4_caf36299-408c-4b4c-a5f3-f6125324fc7e/copy-manifests/0.log"
Sep 17 14:51:17.184250 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:17.184218    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5cb7bb5755-dvn4b_efa8a65a-f3ef-4179-9891-e8b71eb14708/manager/0.log"
Sep 17 14:51:17.284744 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:17.284711    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn_d76458e2-4b38-4f8b-947a-839acc027b9e/istio-proxy/0.log"
Sep 17 14:51:18.103421 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:18.103385    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-znmmb_612ce2f4-8e66-49cd-ad34-acab04c857d2/istio-proxy/0.log"
Sep 17 14:51:18.108939 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:18.108909    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-znmmb_612ce2f4-8e66-49cd-ad34-acab04c857d2/istio-proxy/1.log"
Sep 17 14:51:18.132730 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:18.132695    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-9x9j4_8afff3a4-0475-4e19-96a8-562397330290/payload-pre-processing/0.log"
Sep 17 14:51:18.192333 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:18.192302    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-68854cb984-hm4cb_1414b9c2-7c96-4383-9947-e7abcd68a074/router/0.log"
Sep 17 14:51:19.613457 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:19.613424    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cp7xpn_d76458e2-4b38-4f8b-947a-839acc027b9e/istio-proxy/0.log"
Sep 17 14:51:20.406016 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:20.405978    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-znmmb_612ce2f4-8e66-49cd-ad34-acab04c857d2/istio-proxy/0.log"
Sep 17 14:51:20.410152 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:20.410091    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-znmmb_612ce2f4-8e66-49cd-ad34-acab04c857d2/istio-proxy/1.log"
Sep 17 14:51:20.451666 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:20.451634    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-9x9j4_8afff3a4-0475-4e19-96a8-562397330290/payload-pre-processing/0.log"
Sep 17 14:51:20.503317 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:20.503289    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-68854cb984-hm4cb_1414b9c2-7c96-4383-9947-e7abcd68a074/router/0.log"
Sep 17 14:51:22.661843 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:22.661756    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-wxbtg_9857487e-aa03-45ac-88e4-e6a40b699fef/kube-storage-version-migrator-operator/1.log"
Sep 17 14:51:22.662987 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:22.662956    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-wxbtg_9857487e-aa03-45ac-88e4-e6a40b699fef/kube-storage-version-migrator-operator/0.log"
Sep 17 14:51:23.971219 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:23.971188    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xflkb_82aa9496-50ec-4a83-9d0c-d227df61037a/kube-multus-additional-cni-plugins/0.log"
Sep 17 14:51:23.992272 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:23.992230    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xflkb_82aa9496-50ec-4a83-9d0c-d227df61037a/egress-router-binary-copy/0.log"
Sep 17 14:51:24.016574 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:24.016544    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xflkb_82aa9496-50ec-4a83-9d0c-d227df61037a/cni-plugins/0.log"
Sep 17 14:51:24.038054 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:24.038021    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xflkb_82aa9496-50ec-4a83-9d0c-d227df61037a/bond-cni-plugin/0.log"
Sep 17 14:51:24.058311 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:24.058278    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xflkb_82aa9496-50ec-4a83-9d0c-d227df61037a/routeoverride-cni/0.log"
Sep 17 14:51:24.079239 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:24.079203    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xflkb_82aa9496-50ec-4a83-9d0c-d227df61037a/whereabouts-cni-bincopy/0.log"
Sep 17 14:51:24.099286 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:24.099259    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xflkb_82aa9496-50ec-4a83-9d0c-d227df61037a/whereabouts-cni/0.log"
Sep 17 14:51:24.154228 ip-10-0-137-53 kubenswrapper[2582]: I0917 14:51:24.154197    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-t4548_6504733b-af2d-45a5-a817-7fe57f23d503/kube-multus/0.log"